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ABSTRACT 



Recent research points to two types of strategies for 
preventing substance abuse and behavioral health problems among children and 
adolescents. The first approach emphasizes tobacco, alcohol, and drug 
resistance skills; problem-solving/decision-making training; stress 
management; and modification of attitudes and norms that encourage substance 
use and violence. The second involves the coordinated use of multiple 
societal institutions, such as family, community, and schools, for delivering 
programs. The Pennsylvania Student Assistance Program (SAP) promotes both of 
these approaches through consistently enforced policies at school, as well as 
early identification, intervention, and support to students and their 
families. An evaluation was conducted to investigate six areas of SAPs and to 
help develop a framework to guide future evaluations. The 6 areas are: (1) 
referral process; (2) parent consent practices; (3) intervention services; 

(4) contract provider assessments; (5) school-based probation officers 
involvement; and (6) improvement and satisfaction. The primary findings 
indicate that the SAP teams are working; that parents are involved; that 
services are being provided both in school and in the community; and that the 
students involved in SAPs are showing definite improvement. Further analyses 
that would link process with outcome are recommended for future study. 
(Contains 38 references.) ( JDM) 
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SECTION ONE 



INTRODUCTION 

Recent research points to two types of strategies for preventing substance abuse and 
behavioral health problems among children and adolescents (General Accounting 
Office, 1997). The first approach emphasizes tobacco, alcohol and drug resistance 
skills, problem-solving/ decision making training, stress management, and modification 
of attitudes and norms that encourage substance use and violence. The second involves 
the coordinated use of multiple societal institutions, such as family, community, and 
schools, for delivering programs (Dryfoos, 1990; Hawkins, 1992; Silvia & Throne, 
1997). The Pennsylvania Student Assistance Program promotes both aspects of these 
two approaches through consistently enforced policies at school, as well as early 
identification, intervention and support to students and their families. Further, for those 
students who need them, SAP teams provide linkages with the larger child-serving 
systems, namely the mental health and drug and alcohol assessment and treatment 
providers located in the community (DiRienzo, 1990). 

Why Student Assistance Pro2rams? 

A century ago, barriers to education included infectious disease and untreated physical 
defects that put students at risk for school failure. Today, most of these barriers can be 
addressed in whole or part with immunization, antibiotics, eyeglasses, and other medical 
treatment. However, yesterday’s barriers have been replaced by special health 
needs, chronic diseases, and new sets of barriers based in behavior and lifestyle choices. 



The Centers for Disease Control and Prevention (CDC) have found that the following six 
categories of behavior are responsible for 70% of the mortality and morbidity among 
adolescents: (1) behaviors that cause intentional and unintentional injuries, (2) drug and 
alcohol abuse, (3) sexual behaviors that cause sexually transmitted diseases and 
unintended pregnancies, (4) tobacco use, (5) inadequate physical activity, and (6) dietary 
patterns that cause disease (Kann, Collins, Pateman, Small, Russ, & Kolbe, 1995). These 
problems are based in behaviors that can be prevented or changed. These behaviors, 
usually established during youth, are interrelated and contribute simultaneously to poor 
health, education, and social outcomes. 



Major economic, social, and demographic changes that have occurred in recent decades 
have also had a dramatic effect on the health and welfare of children and adolescents 
(Green, 1994). These changes include a decrease in the time parents spend with children, 
the disintegration of families, and a rapid escalation of the number of children and 
adolescents living in poverty. These changes in conjunction with the following data 
indicate the need to identify students who are experiencing barriers to learning. 
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The CDC’s Youth Risk Behavior Survey of 1997 (CDC, 1998) found that 37% of all high 
school students had ridden with a driver who had been drinking alcohol during the 30 
days preceding the survey, 18% had carried a weapon during the preceding 30 days, 79% 
had consumed alcohol, 47% ever used marijuana, and 20% had attempted suicide during 
the 1 2 months preceding the survey. 



Dryfoos (1990, 1994) estimated that 10% of teenagers are at very high risk for dropping 
out of school because of engaging in a variety of risky behaviors, an additional 15% are 
at high risk, and 25% are at moderate risk. The increasing number of poor and at-risk 
students requires schools to contend with more students who have health and other 
problems that are barriers to learning and academic success. Traditional approaches to 
help remove these barriers to learning are no longer sufficient to deal with these complex 
issues. 



Review of the Literature 



School student assistance programs (SAP) began in the early 1980’ s modeled after 
employee assistance programs (EAP) in the workplace. The EAP model linked the 
individual with community treatment and support personnel from the work site by 
identification of problem behavior, assessment, referral, and follow-up (Johnson, 1980). 

SAPs are school-based identification and referral mechanisms for students who 
experience barriers to learning that may result from substance abuse or other behavioral 
health issues (OSAP, 1988). Nationally, SAPs are the preferred model for providing 
early intervention services to students in schools (Institute of Medicine, 1998; Klitzner, 
Fisher, Stewart, & Gilbert, 1992; Moore & Forster, 1993). With programs in place in 
both urban and rural schools across the country, student assistance programs would seem 
to be the model that schools everywhere could follow (Ottenberg, Olsen, & Schiller, 
1985). The organization and procedures of SAPs have been compared to 
multidisciplinary teams used in special education programs (Moore & Forster). 

Three approaches to student assistance programs are currently in use nationally, namely 
the core team model, the externally-based SAP, and the internally-based SAP (Borris, 
1988). In the core team model, trained school professionals receive referrals from 
parents, teachers, counselors, administrators and other concerned professionals about 
student performance and/or behavior. The externally-based SAP employs outside agency 
personnel to facilitate the program for a limited number of hours per week (Dykeman, 
1994). Successful SAP Core teams have been delineated by Herberg, Hughes and Bond 
(1990) as containing the following five components: 

• Formal student identification 

• Staff training in identification of behaviors 

• Staff involvement in identification of behaviors 

• Intensive training for SAP team members 

• Referral for assessment for treatment and follow-up aftercare support 
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The SAP core team model, while different across schools and states, does appear to be 
the one that is most widely implemented. It adheres to the basic components of formal 
student identification, staff training in identification of behaviors, staff involvement in 
identification of behaviors, intensive training for SAP team members, referral for 
assessment for treatment and follow-up aftercare support, parent involvement, and 
community agency and government human service program collaboration (Herberg et al., 
1990; Moore & Forster, 1993). SAP with its history and experience in schools working 
with students, parents and communities can provide leadership and guidance for how 
schools can help students. 

Carefully selected assessment procedures can quickly and validly evaluate severity of 
substance dependence, adverse consequences, contributing roles of further emotional and 
behavioral problems, cognitive and environmental stimuli. These variables all have 
major significance in suggesting the intensity and nature of intervention needed (Allen, 
Columbus, & Fertig 1995). Assessment, however, also yields valuable secondary clinical 
benefits (Allen & Mattson, 1993). For example giving clients individualized feedback on 
test results may enhance their motivation for change (Miller & Rollnick, 1991) and help 
them formulate personal goals for improvement. Also, research indicates that clients 
themselves highly value assessment (Sobell, 1993) and that programs with formal 
assessment procedures are better able to retain clients in treatment (Institute of Medicine, 
1990). 



Many types of screening instruments are available to providers. The first type includes 
self-report questionnaires and structured interviews; the second type includes clinical 
tests, which can detect substance abuse. Both types of screening instruments should be 
valid (that is, measure what the clinician is attempting to measure) and should yield 
reliable results (that is consistent across rates and time) (National Institute on Alcohol 
Abuse and Alcoholism 1990). 



The Adolescent Problem/Profile Severity Index (APPSI) is a semi-structured interview 
that was developed to fill the need for a reliable, valid and standardized instrument for a 
periodic evaluation of adolescent substance abuse. The APPSI uses a multidimensional 
approach of assessment as an age-appropriate modification of the Addiction Severity 
Index. It yields 70 ratings in 7 domains: psychoactive substance use, school or 
employment status, family function, peer-social relationships, legal status, and 
psychiatric status. The Substance Abuse Subtle Screening Inventory (SASSI) is designed 
to identify people who are chemically dependent and to distinguish them from those 
whose drug or alcohol use may be problematic, but who have not reached the point of 
psychological or physical dependency. The following instruments were collected both 
from the MH and D&A liaisons, provider and team member focus groups at the PASAP 
Conference, as well as the SAP team member and administrator surveys. 



Evaluations of Student Assistance Programs by State Departments of Education have 
recently begun to appear in the literature. For example the Departments of Education in 
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Tennessee (Hepler & Renfro, 1999) and Kentucky (Bryant & Johnson, 1999) are in the 
process of completing evaluations of student assistance programs in their respective 
areas. Scott, Surface, Friedli, and Barlow (1999) completed a study of schools in 
Nebraska, comparing schools with and without a SAP program. They found the schools 
with the SAP programs had lower alcohol use and higher achievement among students. In 
Pennsylvania, DiRienzo (1990) described the development of the Pennsylvania model for 
student assistance. Kelly and Peters (1989) evaluated the implementation of SAP in 
Pennsylvania. Kelly and Peters found that effective implementation of SAPS depended 
upon a number of factors including policy development, administrative and district 
support, effective training, careful selection of team members, communication with 
parents and students about the role of SAP, and developing working relationships with 
community agencies. Initial SAP training in Pennsylvania was evaluated by Swisher, 
Baker, Barnes, Doebler, Hadleman, and Kophazi (1993). Swisher et al. found that SAP 
training programs had been positively received by participants. The implementation of 
SAP core teams was successful, yet imperfect. Lack of awareness about the program 
among parents and students and monitoring the progress of students referred to outside 
agencies were seen as the weakest links. 



History of Student Assistance Programs in Pennsylvania 

Beginning with the Pennsylvania Department of Health’s Office of Drug and 
Alcohol Programs (ODAP) funding of pilot Student Assistance Programs in 1984, and 
continuing to the present, activity at the state and county levels have ensured that students 
experiencing substance abuse and behavioral health problems are identified, assessed and 
linked to supportive services in the community and school. Many organizations and 
agencies are committed to SAP. Their efforts have included the funding of county drug 
and alcohol (D&A) and mental health (MH) agencies to provide linkages with 
community-based organizations and the establishment of a statewide advisory committee 
to insure program quality and increased parental participation. Table 1.1 highlights the 
major events since 1984 that have shaped and defined SAP practice as currently 
implemented in schools and communities throughout the Commonwealth. A detailed 
history of the Pennsylvania Student Assistance Program is available at the Pennsylvania 
Department of Education Student Assistance Program website at http://www.Ksys- 
inc.com/sap/. 
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TABLE 1.1 



Pennsylvania Student Assistance Program Timeline 



1984 


Pennsylvania Department of Health’s Office of Drug and Alcohol Programs 




provides a grant to pilot and implement Student Assistance Programs at 
21 schools in the state. 


1984 


Statewide SAP Advisory Committee representing schools, communities and 
parents is established to provide input for maintaining program quality. 


1985-1986 


Pennsylvania Masonic Foundation for Children establishes funding for SAP 
training and the Commonwealth SAP Training System. 


1986-1987 


Pennsylvania Department of Welfare, Office of Mental Health provides 
funding to expand the SAP services to students at risk of suicide. 
Furthermore the Departments of Health and Welfare increase their financial 
support to county drug & alcohol abuse and mental health agencies to work 
with schools. 


1987 


Pennsylvania Department of Education institutes an initiative whereby 
districts receiving Drug-Free Schools and Community funds are required to 
have at least one SAP Team within each secondary school. 


1989 


Pennsylvania Department of Education begins collecting information 
annually from schools regarding the students who have been processed by 
SAP. 


1990 


Act 21 1 requires that a plan be put into action to assist school districts in 
establishing and maintaining programs of support services and counseling for 
students with drug and alcohol problems. 


1990 


Pennsylvania Juvenile Court Judges Commission initiates and funds the 
school-based probation officer program. 


1991 


County SAP Coordination Teams are established in each Pennsylvania county 
to promote quality SAP services. 


1996 


Regional forums conduct by the Statewide Student Assistance Advisory 
Committee collects information from SAP stakeholders to determine what 
SAP teams need from the State to function effectively. 


1997 


Commonwealth SAP Training System updates the SAP training guidelines. 
Training Standards and Competencies for SAP Core team Professionals are 
established 


1998 


Retraining for existing student assistance professionals in parental 
involvement is designed and implemented. 



Currently, all of the Commonwealth’s public middle and secondary schools and many 
non-public secondary buildings have student assistance programs. These programs help 
students overcome barriers to learning so that the students may remain in school, achieve 
academic success and advance to the next grade or graduate. Current Commonwealth 
initiatives include the refinement of an elementary student assistance program, the 
expansion of the Commonwealth SAP Training System, expansion of the school-based 
probation officer program, support to school administrators continues by way of 
executive seminars specially designed for school administrators, and a system for the 
electronic filing of SAP performance data is in place. 
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In Pennsylvania, the SAP process includes four phases: referral, team planning, 
intervention and recommendations, and follow-up and support. Figure 1.1 shows the 
program model. 

FIGURE 1 . 1 Pennsylvania Student Assistance Program Model 




The first phase of the student assistance process involves problem recognition, referral, 
initial fact finding and parent contact and participation. The many common warning signs 
associated with barriers to learning that may be referred to SAP include difficulties with 
attendance, academic performance, discipline and behavior as well as health-related 
concerns. All student assistance program models have a referral process that involves 
educating teachers, administrators and staff on warning signs that indicate some kind of 
learning barrier (Herberg et al., 1990; Milgram, 1998; Moore & Forster, 1993). In the 
core team model, the team gathers objective information from all parties who work with 
the student in an effort to define the difficulty. During this phase of the process, the team 
does some initial fact-finding to ascertain if SAP is the best resource to help the child, 
and if the referral was appropriate. Central to the Pennsylvania SAP process is the 
contact, consent and participation of the parents in the process. The observations of the 
parent are seen as a crucial part of the process. The Basic Education Circular, produced 
by the Department of Education cites: 

It is the parent’s right to be involved in the process and to have full access to all 

school records under applicable state and federal laws and regulations. 

Involvement of parents in all phases of the student assistance program 
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underscores the parent’s role and responsibility in the decision-making process 
affecting their children’s education and is the key to the successful resolution of 
problems (24P.S. 15-1547). 

Parental 1 contact is defined as communication between the school SAP team by 
telephone, letter, or meeting. Parental participation is defined as the active involvement 
of the parent(s) in the decision-making process that affects their child. A major catalyst 
for increased parental contact, consent and participation in the SAP process has been the 
amendment of the Protection of Pupil Rights Act (also known as the Hatch Amendment) 
as designated in the Family Educational Rights and Privacy Act (FERPA). This revision 
prohibits school personnel from engaging in the survey, analysis or evaluation of 
students, when doing so may reveal information potentially embarrassing to the family. 
Hatch provides that parents not only have the right to know significant information that 
pertains to their child’s school experience, they have the right to decide how to help her 
or him (20 U.S.C. Sec.l232g : 20U.S.C. SecT232h .). 

During phase two, the team gathers objective information about the student’s 
performance in school from all significant school persons with whom the student has 
contact. Teachers, counselors, disciplinarians, school nurses, coaches, activity 
moderators may see early warning signs that when addressed, can prevent other more 
serious consequences. A team member then, with parental consent, conferences with the 
student to understand the student’s perception of the problem. The team, after gathering 
this broad base of pertinent information, can make some initial plans to further explore 
and build with the parent during the conference. With parent and student input, an action 
plan is developed that includes strategies for removing the learning barriers and 
promoting the student’s academic and personal success. 

In phase three, the team intervenes and makes recommendations to appropriate in-school 
and community resources that can best help the young person. Intervention is defined by 
Webster as “to come in between, to stop or to modify.” DeRienzo (1990) descibes the 
Pennsylvania SAP as an intervention model that addresses both drug and alcohol, as well 
as mental health issues that may present barriers to learning and school success. A 
variety of in-school services are available that include one-to-one mentoring with a SAP 
team member or guidance counselor, life-skill development groups such as anger 
management, effective decision making and communication skills, as well as 
academic/leaming strategies. In Pennsylvania, SAP uses a broadbrush approach. It does 
not just work with substance abuse or mental health issues, but can and does address 
them, when needed. Community providers of mental health as well as substance abuse 
services provide a liaison, or professional staff member who works in the school for a 
designated amount of time. These professionals provide school based assessments, 
educational support groups and technical assistance to SAP teams (OMHSAS Guidelines 
for MH System Liaisons, 1997). These liaisons also provide a link with the larger 
treatment systems in the community. SAP teams do not provide assessment, evaluation 
or treatment. 

1 In this document, references to “parental” involvement may be interpreted to include adults such 
grandparents, step-parents, and caregivers who play a significant role in the life of a young person. 
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The student assistance team members do not diagnose, treat or refer for treatment; 
but they may refer for an assessment for treatment... Student assistance team 
members are trained to identify problems, determine whether or not the presenting 
problem lies within the responsibility of the school and make recommendations to 
assist the student and the parent. In cases where the problem lies beyond the 
scope of the school's responsibility, it is the team’s responsibility to inform the 
parent of the problem affecting the child's performance in school, provide 
information on community resources and the options to deal with the problem, 
and, where necessary, set up linkages with resources to help resolve the problem. 
For those youngsters receiving treatment through a community agency, the team, 
in collaboration with the parent and the agency, plans in-school support services 
during and after treatment (Basic Education Circular, 24P.S. 15-1547). 



Phase four, support and follow-up, ensures that the student is supported through the 
change process. In addition, should the child require a higher level of care than initially 
recommended, the team can provide the necessary evidence and structure to promote this. 
Support is also provided to parents and staff so that they can work together to promote 
student success. 
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SECTION TWO 



EVALUATION METHODOLOGY 
Aims of the Study 



In July of 1998, the Pennsylvania Commission on Crime and Delinquency (PCCD) 
funded an initiative to evaluate Student Assistance Programs in Pennsylvania. The 
primary goal of the evaluation was to determine the overall efficacy of Student 
Assistance Programs in Pennsylvania. 

This evaluation investigated six areas of student assistance programs and developed a 
framework to guide future evaluations. The six areas were: 

1 . SAP referral processes 

2. Parent consent practices 

3. Intervention services 

4. Contract provider assessments 

5. School-based probation officers SAP involvement 

5. SAP improvement and satisfaction 

The following report sections provide initial descriptive findings as they relate to the 
above mentioned evaluation areas. It also includes a discussion of the conclusions and 
implications for policy, practice and future evaluations. 

Evaluation Methodology and Study Sample 

To meet the comprehensive scope of this evaluation, a multi-modal approach to data 
collection was employed. This was viewed as the most efficacious mechanism for 
examining the effectiveness of SAP in Pennsylvania, as it allowed for the utilization of 
both quantitative and qualitative data gathered from a variety of perspectives. Five data 
collection methods were employed in the evaluation: 

1 . Statewide Survey of SAP Team Members 

2. County Administrator Survey 

3. Focus Group 

4. Site Visit 

5. Pennsylvania Department of Education Statewide Databases 

A. Statewide Survey of SAP Team Members 

A self-administered closed-ended questionnaire was developed and distributed to 
members of SAP Teams throughout the Commonwealth of Pennsylvania. The purpose of 
the survey was to obtain the information regarding the structure of the Student Assistance 
Program in the school, the effectiveness of the program and the challenges encountered 
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from the perspective of the team member. See Appendix 1 for a copy of the survey. The 

33-item questionnaire addressed the following areas: 

• Descriptive characteristics of the respondent, including gender, age, ethnicity, 
highest level of education, number of teams in which the respondent is involved, 
years of involvement in SAP and receipt of SAP training; 

• Descriptive characteristics of the SAP team, including frequency of team meetings, 
number of team members by position in the school and team leadership; 

• Description of the referral process including methods of publicizing SAP in the 
school, frequency of receipt of referrals by type of referral source, referral process, 
structure of and mechanism for receiving and reviewing referrals, reasons for referral, 
referral acceptance criteria, and interface with students and parents; 

• Expected outcomes as a result of involvement in SAP; 

• Types of assessments employed and recommendations for support services; 

• Involvement in the school and in the SAP team by community based service 
provider liaisons and types of services; 

• Team and parent involvement in treatment and aftercare plans; 

• Accessing of services; 

• Issues regarding parent/guardian consent; 

• School probation officer involvement; and 

• Suggestions to improve SAP. 1 

Utilizing a stratified random selection procedure, 162 out of 1400 schools in the nine 

regions of Pennsylvania, as depicted in Figure 2.1, were invited to participate in the 

survey, of which 154 schools actually participated. 

FIGURE 2.1 



PA Network for Student Assistance Services 







1 The questionnaire was developed via Teleform optical scan technology, a PC-based optical scan data 
collection software system, which enhances the quality of data collection and entry. 
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Upon completion of this task, the Pennsylvania Department of Education sent a letter to the 
coordinator in each of the schools informing them of the evaluation and encouraging their 
participation in the survey. A member of the research team subsequently telephoned each of 
the coordinators to answer any questions they may have had regarding the study and to 
verify the number of team members to receive surveys. In addition, a hotline was 
established which allowed team members to call with any questions and to leave 
information regarding the appropriate mailing address and the appropriate number of team 
members. 

The surveys were mailed with specific instructions for completion and return to the research 
team. As a mechanism for ensuring a high return rate, follow-up telephone calls were made 
to each of the SAP coordinators to verify receipt of the survey, encourage their participation 
and thank them for their involvement. This intensive protocol was very effective as it 
yielded a high return rate of 72% of team members and 95 % of teams. 

Table 2. 1 provides a breakdown by region of the number of questionnaires disseminated to 
schools and teams and the number returned. The results are high for this data collection 
method. Most studies employing similar targeted surveys yield an expected rate of return 
from mail surveys is 60% (Price, 1998). 



TABLE 2. 1 Profiling Questionnaires Sent and Returned by Region and Overall 



REGION 


TOTAL# OF 
SCHOOLS IN EACH 
REGION 


SURVEYS MAILED 
# Schools # Surveys 


SURVEYS RETURNED* 
# Schools # Surveys 


ONE 


252 


35 


354 


31 


239 


TWO 


138 


16 


143 


16 


102 


THREE 


191 


25 


295 


23 


’ 201 


FOUR 


30 


5 


57 


'5 


41 


FIVE 


55 


7 


92 


7 


69 


SIX 


117 


17 


173 


17 


130 


SEVEN 


81 


13 


142 


13 


112 


EIGHT 


239 


33 


306 


31 


225 


NINE 


78 


11 


117 


11 


88 


TOTAL 


1181 


162 


1679 


154 


1207 



* The number of SURVEYS RETURNED includes only those surveys that were completed, not the 
unused copies. 
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B. County Administrator Survey 



The second method of data collection consisted of a structured telephone interview of SAP 
county administrators. SAP county administrators are responsible for disseminating the funds 
to be used in providing services to schools, including the funding of provider involvement in 
SAP teams. Other responsibilities include monitoring, planning and training with regards to 
prevention/intervention programs in the schools and in the community. In addition, some 
counties have personnel who are responsible for working directly with the schools and with 
SAP in their preventive and intervention efforts. Consequently, it was deemed essential to the 
comprehensive nature of this evaluation to obtain the administrators’ perspective regarding 
the effectiveness of SAP. 

The Commonwealth of Pennsylvania is comprised of 67 counties, which are administrated by 
45 county/joinder units. In these units, 61 people were identified as having SAP 
administrative responsibilities. In some units one person held the responsibility for SAP. In 
others, two people shared responsibility for SAP. Of the 61 identified individuals, 53 (87%) 
participated in the county administrator survey. These individuals represent 41 (91%) of the 
administrative units. See Appendix 2 for a copy of the survey. The aspects included in the 
survey consist of the following: 

• Descriptive characteristics of the respondent including gender, ethnicity, highest 
level of education, position, responsibilities regarding the relationship to SAP, receipt 
of SAP training; 

• Involvement of specific types of administrators in SAP such as SCA director, 
MH/MR director, and CASSP Coordinator; 

• Types of contractual arrangements regarding SAP services; 

• Types of service providers involved in SAP services; 

• Types of services offered to SAP and to schools, both in school and outside of 
school; 

• Types of assessment models regularly utilized by contracted provider agencies; 

• Requirements of the contracts held with providers, including written specific 
guidelines; 

• Methods of monitoring contracts including reporting requirements; 

• Reasons for services not being accessed and factors that influence parents’ 
decisions regarding not being involved in community services; 

• Effects of parent and student consent on involvement in SAP and suggestions to 
increase student and parent follow-through with assessment and treatment 
recommendations; and 

• Methods of School based probation officer involvement in SAP. 

C. Focus Group 

The third method of data collection consisted of the facilitation of a focus group at the 
Pennsylvania Association of Student Assistance Professionals (PASAP) conference. The aim 
of the focus group was to gather qualitative data concerning the strengths and weaknesses of 
SAP from the perspective of the school professional and of the outside provider. A 
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representative sample of participants was randomly selected from various regions across the 
state. They included school team members, SAP coordinators, school board personnel and 
community providers who are members of SAP teams. Aspects discussed in the focus group 
included the following: 

• Components of an effective SAP model; 

• Challenges encountered by SAP teams; 

• The effect of parent consent requirements on the team’s ability to work with students; 

• Effective mechanisms for identifying, assessing and referring students to and from the 
SAP; and 

• The role of school based probation officers in the schools and on the SAP teams. 

Two members of the research team facilitated the focus group and two additional members 
were responsible for recording the major points of discussion. Upon completion of the focus 
group, summaries were written and are incorporated in the discussion of some of the major 
findings in the evaluation. 



D. Site Visit 

The fourth method of data collection consisted of the implementation of a pilot version of an 
intensive site visit protocol in five schools across the state. Two members of the evaluation 
team conducted each one-day site visit; having two members involved in the site visit allows 
for a balanced perspective. Table 2.2 describes the site visits components and format. 

TABLE 2.2 Site Visit Components 



Individual 


Time 


Format 


Form 


SAP Coordinator or 
designee 


60 minutes 


Structured Interview and 
Discussion 


Program Profile 


SAP Team 


60 minutes 


Focus Group 


Team Member Focus Group or 
Team Member Survey 


MH//D&A Liaisons 


30 minutes 


Structured Interview 


Liaison Structured Interview 


Probation Officer 


30 minutes 


Structured Interview 


Probation Officer Structured 
Interview 


Teachers 


30 minutes 


Focus Group 


Teacher Focus Group 


Administrator 


30 minutes 


Structured Interview 


Administrator Structured 
Interview 



The goals of the site visit were: 

• To discuss the current level of implementation and effectiveness of the SAP program 

• To identify the challenges and obstacles faced by teachers, administrators, school 
districts, agency, community leaders and organizations 

• To understand the challenges and obstacles of SAP key stakeholders, including the 
identification of the developmental issues that arise over time as SAP evolves 
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Five schools were randomly selected according to school type and region to include schools 
that were located in counties where HealthChoices has been implemented and in counties 
where it has not occurred to date. The purpose was to determine the applicability of the 
protocol across types and location, as well as gather information regarding the effectiveness of 
SAP based upon the perspective of an entire team including the administration of the school. 
Upon completion of the site visits, site visit reports were prepared and the results are to be 
incorporated in the presentation and discussion of key findings. 

E. Pennsylvania Department of Education Databases 

The fifth method of data collection consisted of the utilization of a secondary data source, the 
Pennsylvania Department of Education’s 1997-1998 State Performance Report on the Student 
Assistance Program. Data are gathered annually from public, non-public and area vocational- 
technical school SAP teams regarding the services provided to students during the school year 
and are based on the team member’s perspective in reviewing each student who has been seen 
by SAP during the school year. Aspects included in the report include referrals to the team, 
types of services recommended, types of assessments, team knowledge regarding student 
access of services and changes in performance indicators such as academic achievement, 
advancement, attendance and student improvement. These data, which are gathered both 
objectively and subjectively by team members, are entered on scanning sheets and forwarded 
to a central location for data processing. 2 

For the purposes of this evaluation, the data accessed from this performance report consisted 
of the information submitted by each of the schools that participated in the statewide team 
survey. This allows for consistency in samples and the potential for future linkages of 
databases and future analyses. The above-described methods were integrated into a 
framework for addressing each of the areas to be evaluated. Table 2.3 profiles the research 
methods employed in examining each of the areas. This was also incorporated into the plan 
for analyzing each of the data sets. 



2 A limitation of the database is that it does not provide an objective comparison of change in student 
performance over time, which may be attributed to the student’s involvement in SAP. In other words, it 
does not include student’s pre and post SAP performance records. There were also some inconsistencies in 
the numbers reported which suggest that some of the team members may not have been completing the 
forms accurately. 
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TABLE 2.3 Data Collection Methods By Evaluation Area 





Statewide 
SAP Team 
Mail Survey 


Focus 

Group 


County 
Administrator 
Phone Interview 


Site 

Visits 


Statewide 
Performance 
Tracking Database 


1. Effectiveness of the Current Referral 
Process 


X 


X 




X 


X 


la. Likelihood of students being 
referred to SAP 


X 








X 


lb. Percentage of referrals resulting in 
a recommendation for service 


X 








X 


1c. Benefits and disadvantages of the 
different referral models used 


X 






X 


X 


2. Quality of the assessments provided 
by the contracted providers 


X 


X 


X 


X 


X 


2a. Types of assessment models us and 
which are most effective in meeting 
SAP’s goals 


X 


X 


X 


X 


X 


3. Effectiveness of intervention 
services provided 


X 




X 


X 


X 


3a. Whether specific risk areas 
addressed in a comprehensive plan of 
multiple strategies and services 


X 


X 


- 


X 


X 


3b. Contract provider provisions to 
involve school and parents/guardians in 
treatment and aftercare plans 


X 


- 


X 


X 


X 


3c. Whether referred students access 
and complete recommended services 


X 




X 




X 


3d. Desired outcome behaviors and 
whether achieved 


X 








X 


3e. Whether appropriate aftercare 
services are provided by the contracted 
provider and the teams 


X 


- 


X 


X 


X 


4.Effects of student and parent consent 
requirements on SAP involvement 


X 


X 


X 


X 


X 


4a. Methods to increase consent levels 


X 




X 


X 


X 


5. School based probation officer 
involvement on teams 


X 


X 


X 


X 




6. Suggestions to improve SAP 


X 




X 


X 


X 


6a. Current levels of client satisfaction 
with SAP 


X 


__ 


X 


X 


X 



F. Data Analysis 

The following section of this evaluation report consists of a presentation and discussion of the 
findings by area. Given the categorical nature of the data gathered, two types of analyses 
were conducted. 

Profile analysis: used to identify the general pattern of SAP service among 
school districts. It includes basic demographics, referral effectiveness, quality of 
the assessments, service outcomes, service satisfaction and performance 
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measures. Descriptive statistics consisting of frequencies and percentages are 
utilized. 

Comparative analysis: employed to examine the general patterns in the profile 
analysis by comparing and contrasting SAP service recipient groups, that is based 
on region, locality (urban, suburban, rural) and school type Middle School (MS), 
Junior/ Senior High School (J/SHS), High School (HS), Non Public School (NPS) 
and Area Vocational-Technical Schools (AVTS). These comparisons consist of a 
discussion of similarities and differences in patterns and where appropriate, cross 
tabulations. 

Qualitative data collected at the focus group and site visits are also incorporated into the 
discussion, as it helps place particular findings into the appropriate perspective. It also 
includes a discussion of each of the evaluation areas based on the perspective of the different 
data sources, for example how do the county administrators view the issue of parent consent 
in comparison to the team members view of the same issue. To gather this information, data 
were collected from SAP team members and SAP county administrators. 



G. Profile of Study Sample 



School SAP Team Respondents 

The total number of schools participating in the evaluation activities were 154. Tables 2.4, 

2.5 and 2.6 present the participating schools by type, locality, and region. Each table also 
profiles the number of survey respondents for each category. The distribution of the schools 
by type, locality, and region is consistent with the general distribution of schools in the 
Commonwealth. The schools that responded to the survey are predominantly middle and high 
schools. (See Table 2.4). The results are similar by respondent, that is, a larger proportion of 
the sample consists of team members from high schools and middle schools compare to non- 
public and AVT 

TABLE 2.4 Profile by Schools and Team Members (Source: Team Member Survey) 





Survey Respondents 


Schools 




(N = 1207) 


(N = 154) 


Middle School 


36% (428) 


36% (55) 


Junior-Senior High School 


16% (197) 


16% (25) 


High school 


38% (455) 


36% (55) 


Non Public 


7% (78) 


9% (14) 


AVTS 


4% (49) 


3% (5) 
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As indicated in Table 2.5, a larger proportion of schools and team respondents came from 
urban schools and the smallest proportion of the sample was comprised of rural schools. 



TABLE 2.5 Profile of Responses by Locality (Source: Team Member Survey) 





Survey Respondents 


Schools 




(N = 1207) 


(N= 154) 


Rural 


21% (247) 


20% (31) 


Urban 


45% (543) 


45% (69) 


Suburban 


35% (417) 


35% (54) 



As shown in Table 2.6, the regions with the highest number of respondents are 1 and 8. These 
two regions are primarily comprised of schools from the most urban segments of the state, 
Philadelphia and Pittsburgh. 



TABLE 2.6 Profiles of Responses by Region (Source: Team Member Survey) 



Region 


Survey Respondents 
(N = 1207) 


Schools 
(N = 154) 


One 


20% (239) 


20% (31) 


Two 


9% (102) 


10% (16) 


Three 


17% (201) 


15% (23) 


Four 


3% (41) 


3% (5) 


Five 


6% (69) 


5% (7) 


Six 


11% (130) 


11% (17) 


Seven 


9% (112) 


8% (13) 


Eight 


19% (225) 


20% (31) 


Nine 


7% (88) 


7% (11) 



The teams are comprised of a variety of staff who hold diverse positions within the school. 
Descriptive profiles of teams indicate that 75% of the teams surveyed included at least one 
classroom teacher, guidance counselor, building administrator, school nurse, and agency 
liaison. In addition, one third of the responding teams had health/physical education teachers, 
special education teachers, school psychologists, or school social workers as members. One 
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fifth had an art/music teacher, librarian, school-based probation officer, or central 
administrator. 

Profiles of responding team members indicate that approximately two thirds, 65 % were 
female. The responding team members were primarily Caucasian, 91 %, with a small 
proportion of African Americans, 5 %, and 1 % each Hispanic, Native American, and Asian. 
Greater than two thirds have graduate degrees, as 66% indicated that the Masters degree was 
the highest level of education and 3% had Doctorates. 

The majority of the team member respondents were teachers followed by school counseling 
staff and building administrators (See Table 2.7). Representatives from outside agencies 
comprised a smaller proportion of the sample. This may be due to the fact that most of the 
teams have one or two liaisons that sit on the teams. The majority of the respondents, 66 % 
have been engaged in their profession for greater than 10 years. In addition, they are evenly 
divided with regard to experience on the SAP team, as 43 % have served 1-5 years and 41% 
have been involved for greater than five years. In addition, the majority of respondents, 93%, 
report having participated in SAP training. Furthermore, 44% of the individual team members 
reported receiving advanced training with 76% receiving the training between 1996 and 1999. 



TABLE 2.7 Professional Position in School (Source: Team Member Survey) 



Position 


Percentage 
(N = 1185) 


Teacher 


42% (496) 


School Counselor, Social Worker or Psychologist 


22% (266) 


Building Administrator 


11% (133) 


Agency Liaison/Agency Staff 


9% (101) 


School Nurse 


8% (92) 


Other 


5% (62) 


Librarian 


1% (13) 


School Based Probation Officer 


1% (10) 


Central Office Administrator 


1% (12) 



County Administrator Sample Profiles 

Of the 61 identified SAP county administrators, 53 (87%) participated in the telephone 
interview. Approximately 72% of the respondents are female and the majority are Caucasian. 
In addition, the majority are college educated, with 47% holding a masters degree or higher. 
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TABLE 2.8 Profiles of Responses by Region (Source: County Administrator Survey) 



Region 


Survey Respondents 
(N = 53) 


One 


9% (5) 


Two 


17% (9) 


Three 


17% (9) 


Four 


4% (2) 


Five 


8% (4) 


Six 


15% (8) 


Seven 


9% (5) 


Eight 


11% (6) 


Nine 


9% (5) 



Table 2.8 provides a breakdown of responses by region. Regions 2 and 3 had the highest 
number of participants in the survey and region 4 had the lowest. This may be a function of 
the number of counties in a region, for example, region 7 is comprised of 12 counties while 
region 4 consists of 5 counties. 

Given the small number of respondents from each of the regions, they have been clustered 
into four areas, North, Central, Southwest and Southeast. Table 2.9 profiles the number of 
respondents from each of these areas. 



TABLE 2.9 Profiles of Responses by District (Source: County Administrator Survey) 



District 


Survey Respondents 
(N=53) 


North (SAP Regions 2 and 9) 


26% (14) 


Central (SAP Regions 4, 5, and 6) 


26% (14) 


Southeast (SAP Regions 1 and 3) 


26% (14) 


Southwest (SAP Regions 7 and 8) 


21% (11) 



Due to the smaller sample (N=53) of the administrator survey, the SAP regions were 
combined into four districts for further analysis. Table 2.9 provides a breakdown of responses 
by district. A listing of the counties found in each district can be found in Appendix Four. 

A larger proportion of the respondents had been in their current position for less than five 
years, (42%) with another 30% holding the position of 1 1 years or greater. In addition, 
approximately one half, 49% had been involved with SAP for 6-10 years. Less than two 
thirds, 60 percent, received SAP training from a Commonwealth approved SAP trainer. 
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TABLE 2.10 Primary Relationships with SAP Teams (Source: County Administrator 
Survey) 



Type of Responsibility 


Percentage 
(N = 53) 


Monitoring 


89% (47) 


Planning 


89% (47) 


Fiscal 


66% (35) 


Training 


42% (22) 


Other 


38% (20) 



As indicated in Table 2.10, the SAP county administrators are responsible for multiple 
functions, with the predominant ones being monitoring of contracts, planning of services 
and fiscal issues. In addition, eleven of the counties provide direct service to the SAP 
teams. 
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SECTION THREE 



STUDENT ASSISTANCE REFERRAL PROCESS 



The referral process is the first phase or entry point for the Pennsylvania Student 
Assistance model. Five aspects of the referral process were investigated: (1) how the 
SAP referral process is communicated to potential referral sources (for example, school 
staff, community members, parents/guardians and students), (2) mechanisms for making 
and accepting referrals, (3) feedback to individuals making referrals, (4) types of 
problems referred to the SAP, (5) expected and reported outcomes after being referred to 
a SAP program. 



How the SAP Referral Process is Communicated 

Teams utilize a range of communication methods to disseminate SAP information and 
materials to teachers, students, administrators, agency staff, and community members. 
Table 3.1 shows the frequency team members report using different methods to 
communicate SAP information and materials. Faculty in-service, student handbook, 
brochures, faculty handbooks, and classroom visits are the five top methods utilized to 
communicate the SAP referral process to potential referral sources. 

TABLE 3.1 Communication Methods to Disseminate SAP Referral Information 
(Source: Team Member Survey) 



Communication Method 


Percentage 
(N = 1207) 


Faculty In-Service 


76% (916) 


Student Handbook 


73% (876) 


Brochure 


63% (765) 


Faculty Handbook 


45% (545) i 


Team Member Classroom Visits 


40% (484) 


Newsletter 


39% (465) 


Letter 


36% (434) 


Assembly 


34% (405) 


Mailbox Flyer 


26% (319) 


Parent Handbook 


24% (290) 


Newspaper 


8% (99) 


Community Training 


6% (71) 


Website 


5% (55) 


Cable Television Station 


3% (32) 
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Analysis by region, locality, and school type reveal no differences in the frequency or 
ranking of the top five methods, however variations were reported in other 
communication methods. For example, community training ranged from 0% (Region 4) 
to 15% (Region 9). Website usage ranged from a low of 0% (Regions 4 and 5) to a high 
of 13% (Region 9). Assemblies were most used by Region 5 (5 1%) and least used by 
Region 7 (20%). The suburban and rural schools utilized community training at 9% and 
6% respectively with urban schools utilizing community training 3%. Website usage was 
highest in the rural areas 8% compared to 4% at both urban and suburban locations. The 
variation in choice of methods after the top five suggests that teams use methods which 
most fit their structure, environment and available resources. 

Comparing school type, community training was non-existent for AVTS (0%), and most 
utilized at the J/SHS (12%); 74% of the AVTS used assemblies to communicate the 
process as compared to 24% NPS, 25% J/SHS, 31% HS and 38% MS. 

Analysis of the data also shows that team members use a variety of methods to 
disseminate SAP referral process information. Two or more communication methods 
were used by 92% of the members. While 70% employ four or more communication 
methods. The high percentage of respondents who use more than one method of 
communicating the referral process indicates the importance placed on awareness of the 
SAP process. 



Mechanisms for Making and Accepting Referrals 
A. Making Referrals 

Four mechanisms for making referrals to the SAP program were examined: completion of 
referral form, formal meeting, telephone conversation and informal meeting. Figure 3. 1 
illustrates the utilization of the four methods. 



FIGURE 3. 1 Mechanisms For Making Referrals (Source: Team-Member Survey) 




□ 90% Referral Form Completed 
■ 70% Informal Meeting 

□ 59% Telephone Conversation 

□ 52% Formal Meeting 



Most team members (78%) indicated that they use two or more mechanisms for making 
referrals and 59% use three or more mechanisms. The usage of a referral form as the top 
mechanism for making referrals was consistent across all regions, localities, and school 
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types. Formal meetings were most used in Region 5 (64%) and least used in Region 7 
(28%) and were more likely to occur in the MS (62%) and least likely to occur in AVTS 
(33%), with comparable usage among J/SHS (43%), HS (49%) and NPS (46%). 

B. Accepting Referrals 

The referral acceptance process was examined by two variables. First, who in the SAP 
Program makes the decision to accept a referral and second, what, if any, criteria are used 
to accept the referral. Table 3.2 illustrates who makes the decision to accept a referral. 

TABLE 3.2 Decision to Accept a Referral (Source: Team Member Survey) 



Decision Maker 


Percentage 
(N= 1207) 


Entire Team 


84 % (1016) 


Building SAP Coordinator 


24% (292) 


Team Member - School Counselor 


23% (277) 


Team Member - Teacher 


18% (215) 


Administrator 


16% (194) 


Team Member - Nurse 


13% (160) 


Team Member - School Social Worker 


6% (70) 


District SAP Coordinator 


3% (37) 


Other 


2% (19) 



The entire team was the most used entity (84%) for accepting a referral across all regions, 
localities and school type. The second highest choice was the Building SAP Coordinator 
(24%). Variations in the use of the coordinator as the decision maker ranged from 36% 
(Region 8) to 1% (Region 5). The other regions reported utilizing a building coordinator 
19-27% of the time. 

The second variable examined in relation to the referral process was what type of 
acceptance criteria the teams utilized. Team members were asked about four acceptance 
criteria options: Informal, Formal, Not Sure, or None. Table 3.3 indicates the 
frequencies that teams use each of the referral acceptance criteria. 

TABLE 3.3 Criteria Acceptance Frequencies (Source: Team Member Survey) 



Criteria Type 


Percentage 
(N= 1 1 66) 


Informal 


55% (641) 


Formal 


28% (327) 


Not Sure 


10% (117) 


None 


7% (81) 
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The majority of the team members (55%) indicated that their team has informal SAP 
acceptance criteria, 28% have formal acceptance criteria, 7% have no acceptance criteria 
and 10% of the team members were not sure if the school has any criteria for accepting 
referrals. The usage of formal and informal criteria remained consistent throughout all 
regions; however, the schools with no acceptance criteria varied with Regions 4 (3%), 5 
(3%), 8 (4%), 1 (5%) and 7 (6%) ranging from 3-6%, and Regions 3 (8%), 6 (10%), 9 
(13%) and 2 (17%) ranging from 8-17%. There was no difference in the existence of any 
acceptance criteria or method used among localities of schools. By school type, AVTS 
(41%) and NPS (36%) were more likely to have formal acceptance criteria, than J/SHS 
(22%), MS (27%) or HS (29%). 

Feedback 



The continuation of an effective referral process is often contingent upon the referrer 
receiving some indication that his or her referral has been received. Table 3.4 shows 
what feedback is given to people making a referral to SAP. 

TABLE 3.4 Feedback to Referring Source (Source: Team Member Survey) 



Feedback Method 


Percentage 
(N= 1207) 


Informal Verbal Contact 


70% (848) 


Standard Letter 


38% (453) 


Telephone Call 


17% (199) 


No Feedback Given 


10% (119) 


Other 


6% (76) 



The representative input from the focus groups conducted at the PASAP conference 
indicated that communication between the team and the referrer and the visibility of the 
SAP referral process contribute greatly to the effectiveness of the SAP. Additionally, 
anecdotal reporting gathered from the site visits indicated the importance the faculty 
place on contact with the team in making and following up referrals. 

Types of Problems Referred 

In examining the efficacy of the referral process, it is necessary to review the types of 
problems that respondents indicate the school SAP teams work with. Table 3.5 presents 
problems and barriers to learning that SAP teams are asked to address. 
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TABLE 3.5 Problems Addressed by SAP Team (Source: Team Member Survey) 



Problems Addressed 


Percentage 
(N= 1207) 


Behavior Concerns: MH Related 


94% (1139) 


Behavior Concerns: Drug & Alcohol Related 


93% (1118) 


Suicidal Ideation: Gesture or Attempt 


91% (1096) 


Violated School Policy: Drug & Alcohol Related 


90% (1083) 


Parent/Guardian Reported Problems 


89% (1078) 


Violated School Policy: Non-Drug & Alcohol 


81% (973) 


Self-Reported Problems 


80% (967) 


Drops in Grades 


79% (948) 


Attendance Problems 


78% (946) 


Recovering/Retuming to School 


74% (891) 


Violated School Policy: ViolenceAVeapOn 


64% (776) 


Learning Problems 


49% (595) 


General Health Problems 


48% (573) 


Behavior Concerns (Other) 


19% (223) 



Overall, survey results indicate that 94% of the teams worked with mental health 
behavioral concerns, 93% worked with drug and alcohol behavioral concerns, 79% 
worked with students experiencing a drop in grades, 78% worked with students with 
attendance problems and 49% of the teams worked with students experiencing learning 
problems. 



Expected and Reported Outcomes of SAP 

To understand team members perceptions of student outcomes, the team members were 
asked what outcomes they expected from students’ SAP involvement. Table 3.6 shows 
the SAP team members’ expected outcomes of SAP student involvement. The top three 
outcomes as a result of SAP involvement are the team provides increased 
communications with family (57%), no additional drug and alcohol policy violations 
(50%) and reduced disciplinary infractions (48%). The emphasis on increased 
communication with family reinforces the challenges to schools to involve parents early 
and often throughout the SAP process. 
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TABLE 3.6 Expected Outcomes (Source: Team Member Survey) 



Expected Behavior Changes 


Percentage 


Team Provides Increased Communications with Family (N = 1152) 


57% (651) 


No Additional Drug & Alcohol Policy Violations (N = 1151) 


50% (580) 


Reduced Disciplinary Infractions (N = 1 161) 


48% (556) 


Next Level, Promotion or Graduation (N = 1149) 


47% (541) 


Improved Attendance (N = 1 1 70) 


44% (512) 


Reduced Suspensions (N = 1 1 54) 


42% (481) 


Reduced Further Decline In Attendance (N =1160) 


42% (486) 


Reduced Further Decline In Grades (N = 1 1 62) 


41% (477) 


Reduced Tardiness (N = 1159) 


38% (439) 


Reduced Violence/Weapon Policy Infractions (N = 1140) 


38% (438) 


Grade Improvement (N = 1170) 


37% (432) 


Reduced Incidents Of Class Cutting (N = 1 152) 


36% (420) 


Team Provides Classroom Instructional Strategies (N = 1 146) 


^23% (265) 
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SECTION FOUR 



PARENT CONSENT PRACTICES 

Parental contact, consent, and participation are a priority in student assistance programs in 
Pennsylvania. General practice for SAP teams suggests that parental consent be obtained prior to 
a student’s full participation in the process. Examined in this evaluation were five aspects of 
parent consent: (1) definition of parent contact and participation, (2) methods to obtain parental 
consent, (3) team member responsible for obtaining the consent, (4) written informed consent 
practice for SAP, and (5) parent participation including why parents refuse SAP services. 

Parent Contact and Participation 

SAP parent contact is defined by the Statewide SAP Performance Report as parent and team 
contact via a contact letter, telephone call or face to face meeting at the time of referral or 
immediately following information gathering. The majority (86%) of SAP team members 
reported that parents were contacted when their child was referred to the SAP. Figure 4. 1 
illustrates this point. 

FIGURE 4. 1 Reported Parent Contact (Source: Team Member Survey) 




86% 

Once contacted, the question is how many parents get involved in the SAP process. Participation 
is defined as active communication in the decision-making process between parents, students and 
the SAP team. The Statewide Performance Database reported that over half (55%) of 
parents/guardians participate in the process, as shown in Figure 4.2. At this point in the 
evaluation it is not possible to make firm conclusions; however, the results indicate that over 
two-thirds of parents are contacted and a majority are involved in the SAP process. 

FIGURE 4.2 Reported Parental Participation (Source: Statewide Performance 

Database) 

Participation 
55% 
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In an attempt to understand the process team members use to contact and involve parents in SAP, 
the point at which parents are contacted was investigated. Overall, across the Commonwealth, 
45% of team members reported parent contact after information gathering from teachers. 
Information gathering is defined as assembly of objective, verifiable information about a student 
such as academic, attendance, health and discipline reports, as well as teacher observations. A 
high percent of team members (31%) contacted parents upon referral to SAP. Finally, 1 1% of 
team members reported parent contact prior to conducting a student interview. Table 4.1 shows 
the breakdown of when parents are contacted. Some variations are seen across regions. For 
example within Region 4, 40% of the team members indicated that they seek parental consent 
immediately upon referral to SAP, while another 40% preferred to contact parents after 
information gathering. Little variation was reported in when parents were contacted among 
different schools types and localities. 

TABLE 4. 1 Statewide and Regional Breakdown of When Parents are Contacted (Source: 
Statewide Performance Database) 



REGIONS 





State- 

wide 

(N=1146) 


1 

n=229 


2 

n=90 


3 

n=191 


4 

n=40 


5 

n=69 


6 

n=126 


7 

n=lll 


8 

n=211 


9 

n=79 


After information 
gathering from teachers 


45% 

(519) 


53% 

(122) 


54% 

(49) 


40% 

(76) 


40% 

(16) 


52% 

(36) 


55% 

(69) 


33% 

(37) 


43% 

(90) 


30% 

(24) 


Upon referral to SAP 


31% 

(357) 


24% 

(55) 


21% 

(19) 


40% 

(76) 


13% 

(5) 


26% 

(18) 


31% 

(39) 


34% 

(38) 


35% 

(73) 


43% 

(34) 


Prior to student 
interview 


11% 

(120) 


10% 

(22) 


13% 

(12) 


7% 

(13) 


33% 

(13) 


12% 

(8) 


9% 

(ID 


7% 

(8) 


12% 

(25) 


10% 

(8) 


Before information 
gathering from teachers 


8% 

(87) 


7% 

(15) 


8% 

(7) 


8% 

(16) 


10% 

(4) 


7% 

(5) 


3% 

(4) 


13% 

(14) 


6% 

(12) 


13% 

(10) 


Not sure 


3% 

(38) 


3% 

(7) 


2% 

(2) 


2% 

(3) 


5% 

(2) 


3% 

(2) 


1% 

(1) 


9% 

(10) 


4% 

(8) 


4% 

(3) 


Other 


2% 

(25) 


4% 

(8) 


1% 

(1) 


4% 

(7) 


0% 

(0) 


0% 

(0) 


2% 

(2) 


4% 

(4) 


1% 

(3) 


0% 

(0) 



The site visits offered several explanations for the above findings. Reported at the sites was that 
team members prefer to have significant information to offer parents, rather than the somewhat 
incomplete information found in the initial referral report. It seems that teams use this additional 
step to further study the referral to be sure it is appropriate for SAP, not a teacher/student 
conflict, or a referral better served by another school service. 
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Methods Utilized For Obtaining Parental Consent 



The top three mechanisms for obtaining initial parental consent for SAP services were by 
telephone, letter, and meeting with the SAP team. The primary method noted by 82% of the 
respondents was a telephone call, while 66% used a letter and 52% selected holding a meeting 
between the parent and the SAP team. Team members in the survey could select more than one 
point and in practice, may do so, depending on the nature of the particular barrier to learning 
confronting a student. Table 4.2 shows the methods utilized by participating team members for 
obtaining the consent of parents to allow students to receive SAP services. 



TABLE 4.2 Method of Parental Consent for SAP Services (Source: Team Member Survey) 





State- 

wide 

(N=1207) 


1 

n=239 


.2 

n=102 


3 

II 

tsi OJ 
O 


REGIONS 

4 5 

n=41 n=69 


6 

n=120 


7 

n=112 


8 

n=225 


9 

n=88 


Telephone call to parent/ 


82% 


78% 


89% 


85% 


83% 


90% 


77% 


87% 


78% 


89% 


guardian 


(995) 


(187) 


(91) 


(170) 


(34) 


(62) 


(100) 


(97) 


(176) 


(78) 


Letter sent to parent/ 


66% 


52% 


55% 


79% 


44% 


74% 


85% 


57% 


73% 


57% 


guardian 


(796) 


(123) 


(56) 


(159) 


(18) 


(51) 


(110) 


(64) 


(165) 


(50) 


Meeting with parent/ 


52% 


64% 


57% 


51% 


44% 


38% 


42% 


46% 


54% 


42% 


guardian 


(624) 


(154) 


(58) 


(102) 


(18) 


(26) 


(55) 


(52) 


(122) 


(37) 


Home visit 


8% 


2% 


1% 


18% 


12% 


9% 


1% 


2% 


12% 


13% 




(93) 


(5) 


0) 


(36) 


(5) 


(6) 


0) 


(2) 


(26) 


(ID 


Meeting in community 


3% 


5% 


0% 


2% 


2% 


4% 


0% 


9% 


1% 


6% 


agency 


(37) 


(ID 


(0) 


(4) 


0) 


(3) 


(0) 


00) 


(3) 


(5) 


Not sure 


2% 


5% 


0% 


1% 


5% 


1% 


1% 


5% 


2% 


1% 




(28) 


(12) 


(0) 


(1) 


(2) 


(1) 


0) 


(5) 


(5) 


0) 


Other 


2% 


2% 


1% 


4% 


0% 


1% 


1% 


2% 


2% 


1% 




(21) 


(4) 


(1) 


(7) 


(0) 


0) 


0) 


(2) 


(4) 


0) 



No significant differences are seen across Regions for telephone use as a mode of involving 
parents. In Region 5, 90% of team members report contacting parents by telephone, while 77% 
of respondents in Region 6 contact parents for consent by telephone. Nearly 85% of those 
surveyed in Region 6 contact parents for consent using a letter, while 44% of those in Region 4 
chose this response. For team meeting with the parent, 64% of respondents in Region 1 and 38% 
of those in Region 5 report this as an option. 



Some variation was seen among school types with the primary method for obtaining parent 
consent for SAP services being a telephone call, with over 80% of respondents from each school 
type citing this method. Variation was reported for letters with 80% of AVTS using this method 
compared to 37% in the NPS. Over half of the MS, HS and AVTS use a meeting. Meetings 
were less frequent in NPS (46%) and J/SHS (42%). 
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Role of Primary Contact with Parent for SAP Consent 



The team member survey revealed that school counselors, SAP coordinators or case managers 
are responsible for obtaining parent consent for student participation in the SAP process. Case 
managers may be teachers, administrators, social workers, nurses or other educational 
professionals, who have primary responsibility for working with the student and the family 
throughout the SAP process. Overall, 32% of respondents indicated that primary contact is made 
with the parent/guardian by the school counselor, followed by SAP coordinator (24%). Over 
20% of those surveyed reported that the team member/case manager makes contact with the 
parent to obtain consent. Table 4.3 profiles the role of the primary contact person. 

Some variation in primary contact role was seen in the regions. For example in Region 4, 55% of 
respondents cite a clear preference for team member/case manager making the contact for 
consent. Half of those surveyed in Region 9 report that the team member/school counselor 
makes the contact for consent, while only 1 1 % of those responding in Region 4 report that the 
primary contact is made by the counselor. Regions 6, 7, and 8 appear to differ from the statewide 
frequencies, in that 44%, 45%, and 41% of those surveyed suggested that the initial contact with 
parents for consent is made by the SAP coordinator. These differences may be due to the 
structure of the team, whether the SAP coordinator is full-time or part-time, as well as scheduling 
parameters of teacher-team members who serve as case managers. 

TABLE 4.3 Typical Role of Primary Contact with Parent for SAP Consent (Source: Team 
Member Survey) 





REGIONS 

State- 123 4 56789 

wide n=223 n=98 n=195 n=38 n=69 n=124 n=lll n=208 n=78 

(N=1144) 


Team member - School 
counselor 


34% 

(391) 


39% 

(86) 


39% 

(38) 


34% 

(66) 


11% 

(4) 


42% 

(29) 


36% 

(44) 


32% 

(35) 


24% 

(50) 


50% 

(39) 


SAP Coordinator(s) 


24% 

(274) 


7% 

(16) 


17% 

(17) 


11% 

(21) 


18% 

(7) 


1% 

(1) 


44% 

(55) 


45% 

(50) 


41% 

(86) 


27% 

(21) 


Team member - Case 
management 


22% 

(253) 


36% 

(80) 


22% 

(22) 


27% 

(52) 


55% 

(21) 


32% 

(22) 


9% 

(ID 


14% 

(15) 


12% 

(25) 


6% 

(5) 


Team member - Teacher 


7% 

(79) 


4% 

(9) 


4% 

(4) 


14% 

(27) 


8% 

(3) 


9% 

(6) 


3% 

(4) 


0% 

(0) 


8% 

(16) 


13% 

(10) 


Team member - Other 


6% 

(68) 


6% 

(13) 


6% 

(6) 


11% 

(21) 


0% 

(0) 


12% 

(8) 


6% 

(7) 


4% 

(4) 


3% 

(6) 


4% 

(3) 


Not sure 


3% 

(29) 


5% 

(ID 


1% 

0) 


1% 

(1) 


3% 

0) 


1% 

(1) 


2% 

(3) 


5% 

(6) 


2% 

(5) 


0% 

(0) 


Team member - School 
social worker 


2% 

(18) 


1% 

(2) 


0% 

(0) 


1% 

(2) 


3% 

0) 


0% 

(0) 


0% 

(0) 


0% 

(0) 


6% 

(13) 


0% 

(0) 
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Team member - Nurse 


2 % 

(21) 


2% 

(5) 


9% 

(9) 


0% 

(0) 


3% 

(1) 


0% 

(0) 


0% 

(0) 


1% 

(1) 


— $ 


0% 

(0) 


Other personnel 


1% 

(11) 


1% 

(1) 


1% 

(1) 


3% 

(5) 


0% 

(0) 


3% 

(2) 


0% 

(0) 


0% 

(0) 


i% 

(2) 


0% 

(0) 



Some differences were seen by school locality. For example while the primary contact person 
was the counselor in all areas the reported percentages differed: 38% rural, 31% urban, and 36% 
suburban. Likewise, the use of the SAP coordinator to obtain consent also varied. The range 
here was from 28% in suburban schools to 20% in rural schools. Use of case managers was 
highest in urban schools (26%) with rural and suburban schools reporting 20% and 19%. 
Differences by school type indicate that nearly half (47%) of MS team members surveyed 
reported that the school counselor makes the contact for consent, as compared with 13% of the 
AVTS. Thirty percent (30%) of AVTS surveyed rely on the team member/teacher and 28% of 
AVTS respondents indicated consent was obtained by the case manager. When asked for the 
typical or dominant professional that obtains parent consent for SAP services, HS and NPS 
respondents cited that over one-fourth employ the SAP coordinator, school counselor and case 
manager. About 42% of J/SHS respondents utilize the SAP Coordinator, while only 30% depend 
on the school counselor and 11% rely on the case manager to make the contact for consent. 

Written Informed Consent for SAP Services 

The Pennsylvania Department of Education recommends that teams secure the informed written 
consent of parents prior to their children’s SAP involvement. Informed written consent is 
defined as a formalized written document prepared by school professionals explaining student 
assistance services and importance of parental participation. Figure 4.3, shows whether consent is 
obtained from parents. Revealed is that 80% of team members participating in the study reported 
yes, while 12% reported no, with 8% uncertain. Regional differences show that 95% reported 
obtaining written consent in Region 3 as compared with 60% of those surveyed in Region 7, who 
report getting written consent. There is no appreciable difference by locality with 86% of team 
members from rural schools report gamering written informed consent compared with 77% of 
those in urban schools. Comparison by school type shows that 57% of NPS team members 
secure written consent compared with 96% in AVTS and 85% in HS. 




31 



n 



FIGURE 4.3 Written Informed Consent By Region (Source: Team Member 




□ Yes 

□ No 

■ Not sure 



Region 



Over 45% of SAP county administrators report that they have written specific guidelines that 
require agency professionals to obtain written consent from parents before seeing students in 
school, while 36% do not. Twelve percent of those surveyed reported that that they accepted 
oral consent from parents. Of the 45% who have specific guidelines, about 58% indicate that 
they have their own county-specific guidelines for their provider agencies, while 42% use those 
of another entity. There is broad variation between county regions on this issue, with 100% of 
those counties designated North using only those generated within their area, while 67% of 
those in the Southwest using county specific guidelines. The Southeast was evenly divided, 
while the central district primarily used those of another organization. When asked whose 
consent guidelines were used, 56% relied upon the PA Department of Education SAP 
Guidelines, while 33% reported using Bureau of Drug and Alcohol Program Treatment 
Guidelines. Interestingly, only 11% reported using the Department of Public Welfare Treatment 
Guidelines. Seventy percent (70%) of those questioned were aware of OMHSAS MH Liaison 
Guidelines, and of those, 84% felt that BDAP should produce a comparable set of Minimum 
Guidelines for D&A Liaisons. 

Seventy-three percent (73%) of these administrators report that they have specific written guidelines to 
obtain written informed consent from parents to see their students in school, 18% report that they accept 
oral consent, while 9% report that they have no guidelines. There is some variation by district, where 
100% of those in the North, 67% in the Central and 50% of those surveyed in the Southwest report that 
they have written county specific guidelines for parental consent for liaison services in school. Of these, 
75% report that these guidelines are county-specific, meaning that they created them for use in their 
jurisdiction, while 25% rely on those of another organization. About half rely on the Family Educational 
Rights and Privacy Act, and half cite the 1966 Mental Health Act to assist in crafting consent guidelines. 

Parental Participation 

Parental participation is defined as active communication between the parent, the student, and 
the SAP team. Participation is distinct from contact, in that the parent is actually an active 
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participant in the decision making process. The top four methods for involving parents in the 
SAP process were: 

• Telephone call 

• Individual meeting with a SAP team member 

• Letter 

• Meeting with the entire team 

Eighty percent (80%) in Region 1 and 76% in Region 8 have parent meetings with the team. 

Over one-half (54%) use this mode in Region 4. Regions 1 and 8 are mostly urban, which 
suggests that the low number of parent/team member meetings held in Region 4 may be due to 
transportation issues as this Region is more rural. Over 70% of those surveyed in Region 2 
report holding parent-student meetings with the entire team, while only 13% report using this 
modality in Region 5. In Region 6, 83% of the respondents report encouraging parental 
participation in the process by letters, while 39% in Region 4 use a letter. 

A small difference exists among respondents from the AVTS, 94% of whom report that 
participation is encouraged by telephone call, 65% suggest this occurs through letters, while 39% 
noted meeting with a SAP team member is used. Only 27% noted that parental participation is 
encouraged through a meeting with the entire team. Nearly twice as many MS, J/ SHS, HS, and 
NPS team members reported that their teams encourage parental participation through meetings 
with a SAP team member. 

SAP county administrators were directly questioned as to whether they believed the requirements 
for parental and student consent increased or decreased student participation. Thirty-eight 
percent (38%) of county administrators felt that student involvement increased, with 30% 
suggesting that student involvement decreased. Over one-quarter (28%) felt that the requirement 
neither increased nor decreased student involvement with SAP and 4% were unsure. Figure 4.4 
shows a wide variation by district on the effect that requirements for student and parent consent 
have on students’ involvement in SAP. More than half, 55%, of Southwestern and 50% of 
Northern administrators clearly indicate that the requirements for student and parental consent 
increase student involvement with SAP, while 57% of Southeastern administrators see that 
student involvement is decreased. More than half, 57%, of Central administrators felt that the 
student and parental consent requirements neither increased nor decreased student involvement 
with SAP. These results may be a function of how liaisons and teams are trained to work with 
parents, or as an effect of the impact of managed care. The only other distinction seen is that all 
SAP county administrators in the Southeast contract out for SAP liaison services. 
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FIGURE 4.4 County Administrators’ Perception of Effects of Parental 
Consent on Student SAP Involvement by Region (Source: County 



Administrator Survey) 
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Viewing student and parental follow-through for assessment with community agencies, 38% of 
SAP county administrators perceived an increase, while 25% saw a decrease. Fifteen percent 
(15%) noted no appreciable difference. The highest reported increase is among administrators in 
the Southwest at 55%. The Northern Region was tied with 43% reporting an increase and 43% 
reporting a decrease. 

During the site visits parent consent practices were also explored. The teams describe a time of 
adjustment to the new recommendations for parental contact, consent, and participation. One 
SAP coordinator felt her role was hampered by the need to get parental permission to enroll 
students in SAP as well as in educational groups. In another school, team members suggested 
the new requirements “slowed-down” their work, that they cannot process cases as quickly, but 
the overall participation of parents in the process is both positive and helpful. In addition, they 
have had to prioritize their cases. One team member said she feels her hands are tied when 
parents deny problems and do not consent to SAP services. The assistant principal on this team 
noted an interesting observation that points to the importance of administrators working closely 
with SAP. She noted: “in cases where parents have denied SAP permission to continue, I will 
usually see them after 2-3 more disciplinary referrals. The parents begin to see that a problem 
exists, and during my interview with them, I suggest SAP again, and get consent from them 
during the meeting.” 

The above team members agreed that the new recommendations took time to integrate, and 
making referrals to the MH and D&A liaison took longer, but at the end of the year, when the site 
visit was conducted, the number of referrals was about the same as last year. One team discussed 
losing members because of the change in requirements since the process was more time 
consuming, and this team did not have a scheduled meeting time during the school day. When 
asked how parental consent rates might be increased over current levels, some team members felt 
more training in dealing with denial, confidentiality issues and more time to make calls, access to 
private telephone and more scheduled meeting time during the school day, devoid of duties were 
discussed. 
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The D&A Liaison to one school team noted that she recommends that parents accompany the 
student to the assessment, which in this area is held at the agency. This agency’s philosophy and 
mission promotes parental participation. An interesting systems conflict emerged in discussion 
with this professional. She cited that Act 63 promises confidentiality to students and if the 
agency cannot/does not get the consent of the student, the agency cannot involve parents in the 
process. She did note that in life threatening emergencies, parents are always contacted. An 
example where this confidentiality standard was broken occurred recently when inhalants (such 
as propane gas) were being abused, and when a pedophile was actively supplying alcohol to the 
student. 

Focus group members suggested that good communication, relationship building, educating and 
networking with parents were the major strategies for increasing parental support and 
participation with SAP. Educating parents, not only about SAP, but about alcohol, tobacco and 
other drugs, mental health issues, and so forth, is seen as an important step in involving parents 
in the SAP process. These strategies need to take place both inside and outside of the school to 
involve more parents. Networking through organizations like the Parent-Teachers Association, 
as well as other grassroots community groups may afford good opportunities for these efforts. 

Participants in the focus group identified a good team as one that can bring the parent into the 
process with an attitude of compassion and empathy. They present themselves as allies and 
resources who are concerned with assisting the parent. In the interest of relationship building 
for the future, SAP teams offer this assistance, even when they know they may not get consent 
the first time. Most members were happy to include parents, but in some cases, consent is 
difficult to obtain, and occasionally may be detrimental to the child. Consent is hardest to get 
when communication is initiated via a mailed consent form only. Some teams represented in 
the focus group include the parent in the process when a conference with the student is held. A 
team member talks with the child about the concerns raised, and offers him or her the choice as 
to who should speak to the parent first. At that time, the parent is contacted by telephone and 
informed of the concern while the child is still in the office. Some respondents feel that parental 
consent is much more likely to occur with this type of contact. 

Reasons for Refusal bv Parents 

Parents and guardians sometimes decline SAP services. Team members reported three reasons: 
parent disagree that there is a problem, parent does not want to give consent, and parent chooses 
to try own solution. The answers were consistent across the regions. Figure 4.5 profiles the 
primary reasons for refusal. 

Ninety percent (90%) of those in Region 2 report that the most frequent reason for parents 
declining SAP services is that they disagree that there is a problem, compared to 73% in Region 
9. In Regions 7 and 8, 75% report that parents disagree that there is a problem. Seventy percent 
(70%) of those in Region 3 state that the parent does not want to give consent, while Region 8 is 
lowest at 50%. Parents also choose to try their own solution. Region 9 reports that 31% do so, 
compared to 43% in Region 8 and 63% in Region 4. 
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FIGURE 4.5 Parent Reason for Refusal of SAP Service By Region 
(Source: Team Member Survey) 
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County SAP administrators were also questioned about parent refusal of services. The 
administrators focused on why parents refuse to allow their children to participate in community 
services. The administrators report that the primary reasons that effect parents’ decision not to 
allow their children to be involved in community services are that parents disagree that there is a 
problem (85%) and that parents choose to try their own solutions (79%). Other factors that 
influence parents decisions not to allow their children to be involved with SAP include parents 
not wanting to sign an informed written consent form, parental disagreement with the SAP 
process and an inability to reach parents after numerous attempts. Figure 4.6 shows a breakdown 
by region of the administrators’ responses in counties/joinders that contract with community 
agencies to provide SAP services. 
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FIGURE 4.6 County Administrators’ View of Factors Influencing Parental 
Refusal of SAP Services By District (Source: County Administrator 
Survey) 
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The only significant variation occurs in the Southwest Region, where 91% of administrators 
suggest that parents wanting to try their own solution was the highest reason given for refusal of 
community services by parents. The stigma associated with mental illness, as noted by a few of 
the administrators, was viewed as a barrier to involvement with SAP. 





SECTION FIVE 



EFFECTIVE INTERVENTION SERVICES 

To appropriately explore the effectiveness of SAP’s intervention services, it is important to 
examine them according to five aspects: (1) to identify the primary reasons students are referred 
to SAP; (2) once a student is referred, to delineate the principal actions taken by SAP, for 
example, is it recommended that a student participate in an in-school program or in a community 
based one; (3) to determine if the students are accessing the services and if the 
parents/guardians/family members are involved in these efforts, both in school and out of school; 
(4) to examine the provision of appropriate follow-up and aftercare services; and (5) to describe 
the desired outcome behaviors and the extent of achievement. 

Types of Problems Referred to SAP Teams 

To determine the effectiveness of the intervention, it is important to understand the types of 
problems referred to SAP teams. As discussed in Section Three, the teams see a variety of 
referrals necessitating a specific level of intervention. The results of the team survey indicate 
that the teams work with a variety of mental health, drug and alcohol behavioral indicators and 
policy violations, both D&A and non D&A. 

When examining the types of referrals broken down by region and locality, the differences were 
negligible. However, comparisons between school types showed wider differences in the 
nonpublic schools when compared to the publics. For example, drug and alcohol issues were 
reported to be seen by 100% of the members of the AVTS teams, followed by 96% from HS; 
however, 65 % of the team members from the responding NPSs report working with AOD 
issues. Suicidality was seen by 100% of the AVTS teams but by only 77% of the NPS teams. 
Over 98% of the AVTS report working with D&A policy violations while 69% of NPS teams 
report working with these violations. About 94% of the AVTS and 92% of the HS teams report 
accepting parent reports/referrals to SAP, while 74% of NPS respondents indicate that they 
accept referrals from parents. With regard to Non-D&A policy violations such as disciplinary 
infractions, 84% of AVTS teams compared to 58% of the NPS teams report such involvement. 
Student self referral to SAP is reported by 98% of the responding AVTS teams and 86% of the 
HS team respondents, while only 51% of the responding NPS team members cite this as a reason 
for referral. 

The data also indicate that there is no real variation between school types for students referred 
for drop in grades, while attendance issues are reported by 85% of the J/SHS teams and 54% of 
the NPS teams. Seventy-eight percent of AVTS teams report referrals for Violations of School 
Violence Policy (weapons) while 60% of MS and HS teams, and 37% the NPS teams surveyed 
report referrals for this issue. 

MH and D&A behavioral concerns are followed by suicidal ideation in the AVTS. There is no 
appreciable difference between MS and HS by type of school. NPSs traditionally see lower 
numbers, with the highest reports for MH and D&A concerns, suicide and drop in grades. 
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The low reports from nonpublic schools suggest that the frequency and nature of problems 
experienced in NPSs are different, or that SAP is integrated into these school settings differently 
from the way it occurs in the public schools. The NPSs are not required to retain students who 
exhibit a variety of different behavioral problems. The AVTS by nature works with an older, 
perhaps riskier, population of students. In addition, AVTS work requires close supervision by 
faculty, more hands-on types of activities where teachers can see a variety of behavior issues that 
may be masked in a more passive, academic learning environment. 



The types of referrals to SAP teams are also documented in the reports submitted to the 
Department of Education. Figure 5.1 provides a profile of the responses. The three top reasons 
for referral are mental health concerns, other concerns, and drug and alcohol concerns 
respectively. Other behavioral concerns consist of generalized behavior problems not easily 
labeled as MH or D&A related. In Pennsylvania, SAP teams are trained to identify behaviors 
that interfere with learning, and not necessarily identify them as MH or D&A related. 

FIGURE 5.1 Primary Reasons for Student Referral to SAP Teams (Source: Pennsylvania 
Department of Education) 
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Similar reasons were identified at the site visits. Interviews with some teachers reveal that 
attendance and decline in performance are the two primary indicators that lead to a referral to 
the SAP team. However, others indicate changes in behavior such as a student seeming more 
depressed or angry. 



Types of Actions Taken by SAP Teams 

Intervention is broadly defined among Pennsylvania SAP teams by the actions taken to promote 
the student’s academic and personal success. The primary actions taken by SAP Teams after 
referral and fact-finding, reported by the Team Survey are described in Figure 5.2. The primary 
types of actions include a recommendation for an assessment, 97%, continued monitoring by 
SAP, 94%, referral to MH and D&A agency, 90% and conference between the school, the parent 
and the student at 88%. 
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FIGURE 5.2 Primary Actions Taken by SAP Teams (Source: Team Member Survey) 
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There is no great variability by region. For example, over 90% or more of regions report that 
they do make recommendations for assessments. The variability occurs when comparing the 
results within regions 5 and 6 where 99% report this type of recommendation; however 91% of 
the responding teams in Region 9 cite this type of recommendation. The action of continued 
monitoring by SAP is mentioned by 98% of region 6 respondents and by 85% from region 4. 
With regard to referral to MH and D&A services the high is 96% in regions 3 and 5, while the 
low is 85% in region 8. All regions seem to follow the same primary steps to assist students and 
their parents/guardians. 

When comparing localities, that is, rural, urban and suburban, there are no appreciable 
differences when making recommendations for assessment or in continued monitoring. 

For MH and D&A referrals, 94% of team member respondents from rural schools conference 
with school, families, and student; while 86% of suburban schools cite this type of action. 

Three fourths, 75% of respondents from urban schools refer to in-school resources while 68% 
of rural schools use this strategy. 

School type comparisons suggest there is no difference in terms of recommendation for 
assessment, or for continued monitoring. A smaller proportion of respondents from NPS teams 
refers to MH and D&A agencies (71%) while 94% from the AVTS report doing so. Responses 
from all types of public schools (MS, J/SHS, HS) are about the same for these domains. 
Conference with school, agency, and parent is lowest in NPSs (50%) and 76% for AVTS. 
Referral to in-school programs is noted by 84% of AVTS respondents, 62 % of the J/SHS 
sample, followed by 60% of the nonpublic school team members. 

Focus group participants indicate that good interventions involve the early identification of 
problems, preparation of a full spectrum of services, and providing good linkages with 
community resources. Teams depend on parents and staff other than team members to make 
referrals, an important step in the identification of problems. Offering a full spectrum of services 
is contingent upon team organization and good utilization of liaisons and community resources. 
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In turn, linkages with community resources and agencies were seen as a major role of a good 
liaison. 



A. Service Recommendations 

Based upon the needs of the student, SAP teams may make recommendations for services within 
the school or within the community. The following sections provide a description of the 
frequency in which specific services are recommended. 

School-Based Services 

Table 5.1 profiles the frequency in which various school-based services are recommended by 
SAP teams. The results indicate that the types of services recommended most frequently, that is, 
often/always are: follow up with a SAP team member or a counselor, each at 52%, followed by 
special groups, either D&A intervention or MH special group, 36% and 33% respectively. 
Approximately one third, 31% of team members indicate that they frequently recommend 
referral for academic support. One on one may be recommended more frequently as it may be 
viewed as a vehicle for addressing an issuing while protecting the privacy of the student and the 
family member. Participation in groups may be perceived as more stigmatizing, as it requires 
sharing of issues with others. It may be that after one on one, students and family members 
would be more comfortable moving into a group milieu. 

TABLE 5.1 In-School Support Services Recommended by SAP Teams (Source: Team Member 
Survey) 



Type of Service 


Often/Always 

(N=1207)' 


1:1 Follow-Up w/Team Member 


53% (615) 


1 : 1 Follow-Up w/Counselor 


51% (591) 


DA Intervention Group 


35% (405) 


MH Special Issue Group 


32% (362) 


Academic Supports 


30% (332) 


Other In-School Group 


20% (224) 


MH Aftercare Group 


18% (207) 


DA Aftercare Group 


17% (197) 


Formal Intervention 


16% (184) 



Community Based Services 

As discussed in Section One, in Pennsylvania, the core team model utilizes trained school 
personnel to identify and intervene with students who are troubled by MH and/or D&A issues. It 
also includes the integration of community agency personnel as liaisons to the team. Liaisons 
are employed by drug and alcohol and/or mental health agencies, or by the county/joinder itself, 
to provide on-site pre-assessments, technical assistance, and a link with the larger systems that 



'The question asked team members to indicate which services they recommend. Consequently, the number of 
responses for each item may be less than 1207 and the percentage is calculated accordingly. 
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they represent (MH Guidelines, 1997). A team’s ability to take action on a referral in an effective 
fashion may be attributed to the amount of involvement time a community liaison has in the 
school. This aspect was addressed by determining the extent to which liaisons are actually 
involved in teams. 

Drug and Alcohol Liaison Services 

Results from the Team Survey indicate that 83.7% of participants stated that they received SAP 
liaison services from a D&A agent. With regards to frequency of involvement in the school, 40% 
of the respondents saw the D&A liaison one full day per week in their buildings. Nearly 32% of 
respondents noted that the D&A liaison spent less than one day per week serving in the school 
while 19% stated that this specialist spent more than one day per week in the building. 

Regional comparisons indicate that of those in Region 5, 95% recount receiving D&A liaison 
services, while 92% of the respondents in regions 3 and 4 report overall receipt of such services. 
Region 8 is the lowest, with only 68% of schools surveyed indicating the receipt of D&A liaison 
services. There were also differences in the receipt of services when comparing school type. 

Fifty six percent (56%) of team members from non-public schools report receiving D&A liaison 
services; while the highest is reported by AVTS respondents at 96 %, followed by J/SHS at 90 
%, and HS and MS at 84 %. 

Regional comparisons of the frequency of receipt of D&A services indicate that 60% of 
respondents from Region 2 reported receiving services one day per week, while only 34% of 
respondents from Region 8 reported receiving weekly services. Regional comparisons focusing 
on receipt of services two or more days per week show that the highest is reported by Region 5 at 
29% and the lowest is in Region 6 at 4%. 

There were no real differences in the frequency of receipt of services when comparing responses 
by type of locality. However, when comparing the responses based upon school type, it was 
found that 84% of AVTS team members report D&A liaison service provision at least one day 
per week. In addition, approximately one half of the J/SHS and the nonpublic school team 
members report receiving services at least one day per week, 52% and 51% respectively. 

Mental Health Liaison Services 

Eighty-three percent (83%) of team member respondents indicated that MH Liaison services are 
provided to their SAP team. Nearly 40% of the team respondents noted the presence of the MH 
liaison for one day per week, and 31% saw the MH liaison less than one day per week. About 
18% noted the presence of the MH liaison more than one day per week, with 1 1% marking 
unsure. 

Regional comparisons indicate the highest percentage reported for MH Liaison services was for 
those respondents in region 8, with 76% noting the presence of the service provider at least 1 
day/week. 2 The lowest was in region 6 with 45% of those surveyed noting that services are 
present at least one day per week. In region 4, 1 1% of the teams report having the liaison 



2 Due to managed care and other issues, many agencies in region 8 now offer both MH and D&A liaison services to 
schools. This may account for the difference in responses with regard to the provision of D&A liaison services 
when compared to the provision of MH liaison services. 
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involvement in the school three or more days per week; while in region 6, one third have MH 
services one day per week, and 12% have them two days per week. Approximately one fourth, 
27% of region 8 respondents report receiving MH services at least 3 days per week 

Locality comparisons yield little variation. However, comparisons based on school type indicate 
that nonpublic schools receive MH liaison services less frequently than the other types of 
schools. For example, 68% on the NPSs receive such services less than one day per week, while 
74% of the HSs respondents indicate receipt of such services at least one day per week and 
almost 25% have MH services three or more days each week. Differences in frequency of 
receipt of service by school type may be a function of school size and of number of schools 
served by a contract provider. 

The focus group participants stated that while most liaisons are not directly involved in the 
treatment of the child, they were responsible for assessments and the coordination of referrals to 
outside agencies. In addition, good liaisons are viewed as supporting the nature and degree of 
visibility of SAP teams within a school by acting as bridges between administrators and in- 
school team members, by maintaining an attitude of support for the student and by linking the 
SAP teams with the larger community. This role is viewed as important both in terms of 
referrals and also in terms of general parental support and community building among agencies. 

B. Recommendations for Community Based Services 

The results in Table 5.2 indicate that the primary recommendations for community based 
services are an assessment by an MH or D&A provider, each at 88%, followed by other social 
service involvement. Regional comparisons indicate that Region 8 reports the lowest frequency 
of recommendation to community services by SAP, 78%, and Region 7 is highest at 96%. For 
locale, there does not appear to be wide variation. Recommendations to community agencies are 
lower, but not substantially, for urban schools (84%) than for rural or suburban schools, each at 
92%. 

TABLE 5.2 Frequency of Recommendation to Community Services "by SAP Teams (Source: 
Team Member Survey) 



Recommended Service 


Percentage 

(N=1207) 


Assessment by D&A provider 


88% (1067) 


Assessment by MH provider 


88% (1060) 


Other social service involvement 


79% (957) 


Juvenile probation/legal system 


40% (488) 



Comparisons by school type indicate that the frequency of D&A or MH recommendations is 
highest in HS (95% and 92% respectively). However, it is lowest in the NPSs, 59% and 68% for 
D&A and MH respectively. Seventy-one percent (71%) of AVTS report making referrals to 
Juvenile Court compared to 47% in HS and 3% in NPSs. The low rate in nonpublic schools may 
also be a function of the types of students who are allowed to remain in them. In some instances, 
students who are experiencing barriers to learning move from one type of school to another. 
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The types of services recommended by SAP teams are also documented in the reports submitted 
to the Department of Education. As depicted in Table 5.3, the primary services recommended 
are MH assessments and D&A assessments. 

TABLE 5.3 Services Recommended by the Core Team 
(Source: Pennsylvania Department of Education) 



Recommended 

Services 


Percentage 

(N=8104) 


MH Assessment 


25% (2026) 


D&A Assessment 


17% (1378) 


Other Social Service Agency 


7% (567) 


Other 


45% (3647) 


Juvenile Probation 


3%( 243) 



The distribution of the types of services recommended as reported to the Department of 
Education is similar to that reported in the statewide survey. However, the proportions are 
smaller which may be a function of the data source. The statewide survey is based upon team 
member perception, while the PDE survey is student based. In other words, they refer to records 
maintained on students who are involved in SAP. 

As previously discussed, a survey of County Program Administrators responsible for the 
provision of SAP-liaison services was conducted to obtain their perspective regarding the role of 
SAP and community based agencies in schools. These services that are provided to schools by 
community based agencies are primarily funded via contracts with the county. They also fund 
agency participation in school SAP team meetings to promote early identification and problem 
solving. It is expected that by their presence in these meetings, liaisons are able to bring 
different perspectives, knowledge, and skills to the team. 

Table 5.4 provides a profile of the types of agencies involved in county contracts for SAP 
Intervention services. The data show that 43 % of the county administrators contract with mental 
health agencies, 36 % with MH and D&A agencies, and 33% with D&A organizations. Other 
types of providers with whom the counties contract are independent consultants and base service 
units for these services. A negligible number noted agreements with school districts and 
intermediate units as licensed providers of these services. 
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TABLE 5.4 Types of Agencies Contracted to Provide SAP Intervention Services 
(Source: County Administrator Survey) 



Contracted Agency 


Percentage 


MH Only 


43% 


MH/D&A Agency 


36% 


D&A Only 


33% 


Independent Consultant 


29% 


Base Service Unit 


29% 


OTHER 


17% 



Services Provided by Contracted Agencies 

Counties, and in some cases the schools themselves, contract with different types of agencies to 
provide a variety of services in schools. Such services include: SAP intervention services, 
participation in SAP team meetings, crisis management and postvention services, in-school 
educational support groups and parent education and consultation. Results of the county 
administrator survey indicate that these services are primarily provided by MH agencies, 
followed by MH/D&A and D&A agencies. 



As indicated in Table 5.5, agencies use more than one assessment model, suggesting that 
students and parents may be given the option as to where they would like to have such an 
assessment occur. The primary assessment model utilized is a one to one assessment with the 
student and parent at the school, 93%. This may be attributed to the presence of a liaison at the 
school at least one day per week, making it easier to provide the service at the school. This may 
reduce travel time and transportation issues. For example, one of the schools that participated in 
the site visit indicated that the agency’s location was one hour away from the school, making it 
difficult for parents and students to travel to the program. The other type of assessment occurs at 
the agency, but this happens in only 38% of the cases. In addition, interventions, when they 
occur, primarily take place at the school. This may be related to difficulties with transportation, 
parent work schedules and the agency’s hours of operation may conflict, or the stigma attached 
to entering a community-based facility. 

TABLE 5.5 Assessment Models Used by Contracted Provider Agencies 
(Source: County Administrator Survey) 



Assessment Modality 


Percentage 

(N=53) 


1 : 1 Assessment w/Student, Parent at School 


93% (39) 


1 : 1 Assessment w/Student, Parent at Agency 


38% (16) 


Intervention Group at School 


33% (14) 


Intervention Group at Agency 


14% (6) 



Services Recommendations 

As discussed in the previous section, one of the functions of the contracted service provider is to 
conduct an assessment to determine the types of services needed by the student and/or family 
and to make a specific recommendation for service. Such information is reported to the 
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Department of Education. The primary service needs based upon the assessments relate to 
mental health, support/aftercare and D&A issues. The majority of services are outpatient in 
nature, as indicated by 16.8 % of the recommendations being made for outpatient mental health 
services compared to 2.7 % for inpatient mental health services. Students with more serious 
conditions may already be receiving help in the community and may not be interfacing with 
SAP. In addition, mental health issues may be more easily identifiable than those related to a 
drug and alcohol problem. 

Factors that May Effect Service Acquisition 

As indicated in the statewide performance database, the majority of students, 61%, receiving a 
recommendation for service actually accessed the services. Greater than one fourth, 29%, did 
not access the services and the remainder accessed some of the services. The primary reason 
given for not accessing services is student refusal, 41%, or parent/guardian refusal, 36%. 

This finding is supported by site visit and focus group information, as many of the team 
members indicated that some families were not interested in accessing services due to the stigma 
of mental illness or the desire to manage the issue without the help of the school or the agency. 

Finally, the results of the statewide survey of SAP team members indicate that parent/guardian 
refusal and student refusal (60% and 54% respectively) are the primary reasons for not accessing 
services in the community. Other reasons cited include transportation problems, 44%, lack of 
insurance, 43%, and cost, 34%. 



Regional comparisons are similar, with the exception of Region 2 in which respondents cite 
student refusal and 69% mention parent/guardian refusal as the primary reasons for services not 
being accessed. Lack of insurance is highest in Regions 1 and 3, at 53% and 52%, 70% of team 
respectively, and lowest in Region 2 at 27%. There are also no significant differences when 
comparing locality. However, school type comparisons indicate that student refusal is reported 
by a larger proportion of respondents from AVTS, 86%, than from MSs or NPSs, 40% and 41% 
respectively. Two thirds, 67%, of the MS respondents cite parent refusal as an important reason 
for services not being accessed, while 47% of AVTS respondents cite this as an issue. 

In examining the reasons for services not being accessed, significant qualitative information 
surfaced from both site visit and focus group participants. Some members articulated that 
managed care does not always operate with the best interests of the adolescent in mind; nor are 
some parents adequately insured. This is seen as a systemic, political dilemma that will have to 
be addressed on a long-term basis. No concrete suggestions were given concerning how best to 
approach this problem. 

Utilization of a Comprehensive Plan to Address Risks to the Student 
Post intervention assessments are conducted by community agency liaisons who work with 
schools. As described above, various assessment models and tools are utilized to address a 
variety of home, school, and community-based issues. This area was not directly addressed in 
the statewide survey; however, indirect evidence from the statewide and county administrator 
surveys, as well as from the site visits, can assist in discussing the role of parents, school and 
community resources used to meet the areas of identified risk. 
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As described in Table 5.5 the primary assessment model includes the parent and student at the 
school. This probably makes it easier for the service provider to engage the student and the 
family member in a convenient location, thereby limiting the likelihood of student or 
parent/guardian refusal in the process. 

This area was explored in more detail at some of the site visits. For example, a structured 
interview of the D&A liaison held during one site visit indicated that as long as students sign a 
release of information, a one-page summary of assessment results is sent to the team. This 
summary only contains a profile of use, risk factors, and recommendations. Interestingly in one 
county, if the county co-funds treatment, the treatment agency is required to send school-based 
strategies which reinforce treatment plans. Private agencies are not required to do this, and 
generally there is little communication between these resources and the schools. If the child has 
been hospitalized, a re-entry conference is scheduled one week prior to the student’s return to 
school. No child in this particular county is denied D&A treatment because of inability to pay. 

One MH liaison noted that when students attend treatment with her agency, she is the link 
between the school and the agency. This liaison finds that working directly with the guidance 
counselor and SAP team, and as a SAP member is most helpful in shaping school-based 
strategies that complement treatment and aftercare goals. If the student is referred and attends 
treatment at another agency, the guidance counselor usually gets the information, which may 
not always get to the team, or the liaison who can provide aftercare support. 

As previously discussed, the primary mode of assessment occurs as a one to one assessment 
with student/parent at school (93%) while 38% report this occurs at the agency. The third most 
preferred mode of assessment is by intervention groups run at the school. There is considerable 
variation by region, with 100% of Southwest and Central regional county administrators stating 
that the 1:1 assessment with the student/parent occurs at school, while only 80% in the North 
and 93% in the Southeast reporting this. 71% of administrators in the Southwest report agency- 
based assessments with student/parent, compared with 50% of those in the North. Only 29% of 
respondents in the Southeast and 18% of those in the Central regions cite this modality. 
Southwest administrators report the highest frequency of intervention groups held at school to 
be 43% while the lowest occurs in the North at 20%. Both Central and Southeast regions cite 
this mode at 36%. 

Acquisition of Recommended Services and School/Parent Involvement 

Overall, SAP teams do have some involvement with agencies, which, with parental consent, 
notify the schools of treatment status and share treatment and aftercare plans. Table 5.6 profiles 
the frequency of involvement of SAP team involvement in treatment and aftercare plans. 

Twenty percent (20 %) of the team members report being involved always/often in notification 
of treatment status and 19% of teams report being involved often/always in treatment and 
aftercare plans with agencies. In addition, approximately one half of the respondents report 
sometimes being involved in these ways. Sixteen percent (16%) of team members surveyed 
report being involved always/often in regularly scheduled agency and school planning meetings, 
while 34% report this occurs sometimes. However, less than one third, 30% state that it never 




47 

57 



occurs. Outside agencies involve SAP teams sometimes, 34%, 29% never and 11%, 
often/always. 



When questioned about the frequency with which outside agencies involve the SAP teams in 
joint meetings with school, students and family members, 47% state this occurs sometimes, 22% 
never and 11% often/always. Weekly telephone conversations occur between agencies and SAP 
teams sometimes, 30%, never 33%, and 6% always/often. One third, 33%, of the respondents 
report monthly telephone conversations between schools and agencies, while 28% state they 
never transpire and only 7% state that this occurs often/always. Interestingly, 46% of 
participants suggest that outside agencies sometimes involve the SAP teams in notification of 
treatment status compared with 20% reporting this takes place always/often. 

TABLE 5.6 Frequency of SAP Team Involvement in Treatment and After Care Plans 
(Source: Team Member Survey) 



Type of Involvement 


Often/ Always 
(N=1207) 3 


Notification of Treatment Status (N=1097) 


21% (233) 


Treatment and Aftercare Plans (N= 1 1 28) 


20% (220) 


Scheduled Agency and Team Planning Meetings (N=l 117) 


18% (198) 


Periodic Plan Review (N=1077) 


12% (129) 


Joint Meetings with School, Student & Family (N=1102) 


12% (131) 


Monthly Telephone Conversation (N= 1 039) 


8% (86) 


Weekly Telephone Conversation (N=1057) 


7% (71) 



When teams are not involved, 56% of teams surveyed report that parent/guardians refuse to give 
consent for this information sharing between the treatment agency and the school. In addition, 

41 % of the respondents believe it is because consent from parents/guardians is not obtained, 
while 29% were uncertain about the lack of involvement. Other reasons for lack of involvement 
are due to the high caseload involvement that makes it difficult for agencies to work with 
schools, 17%, and agency’s possible misinterpretation of confidentiality laws, 9%. 

Regional comparisons indicate some variation regarding this issue. For parent/guardian consent 
not obtained, 49% both in Region 3 and 6 and 24% in Region 4 marked this response. 
Parents/guardians refusal of consent is cited most in Region 3, 71%, while Region 4 is the lowest 
at 37%. One third, 32% of respondents in Region 4 indicate that agencies are unable to work 
with schools due to the volume of the caseload, while this is cited by only 9% of respondents 
from Region 6. There is no appreciable difference with regard to this issue when comparing 
localities. 

Analysis of the data by school type suggests that the most frequent reason for the lack of 
collaboration between schools and treatment providers is consent refusal by parents with the 
78% of AVTS noting this, 56% of MS and HS, 55% of J/HSH and 44% of NPS. Of the schools 

3 The question asked team members to indicate which services they recommended. Consequently, the number of 
responses for each item may be less than 1207 and the percentage is calculated accordingly. 
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surveyed, 48% of J/SHS and 45% of AVTS suggested that parent/guardian consent was not 
obtained, compared with 43% of HS and only 21% of NPS indicating that this is an issue. 

A. Access and Completion of Recommended Services 

One evaluation goal was to determine whether students and families access and complete 
recommended services. As previously discussed, SAP teams make recommendations for 
assessment to licensed drug and alcohol, licensed mental health and other social service agencies 
such as Children Youth and Families, as well as juvenile probation. Thus, it was necessary to 
look at the types of recommendations made, and why services are refused 

Recommendations from Agency Assessment 

Based on those who received a recommendation to community agency services, 18% were 
referred on to further mental health treatment on an outpatient basis, and 9% to other community 
services. Eight percent (8%) of students were directed to outpatient drug and alcohol treatment 
as well as to in-school support and aftercare services. For the 5% of those assessed, no 
community agency referral was given for further treatment while inpatient mental health and 
drug/alcohol treatment were suggested to the 3% and 2% of students, respectively. 

The Statewide SAP Performance Database results indicate that over two-thirds of those assessed 
accessed all or some of the recommended services. When viewing the results, 58% are noted to 
access all recommended services, 12% access some of the recommended services, while 30% do 
not access recommended services. When students and parents do not access the recommended 
services, the most frequently cited reasons are parent and student refusal. 

B. Failure to Access Recommended Services 

Both SAP county administrators and surveyed SAP team members noted that the two major 
reasons that students do not access recommended community services are parent/caregiver 
refusal and the student’s refusal. SAP county administrators were also concerned that 
transportation problems and denial of coverage by health insurance also play a role in the failure 
to access services. As presented in Figure 5.3, about one-third (32%) of administrators noted 
that scheduling conflicts are an issue, while 28% noted that waiting lists prevented students 
from accessing recommended services. 
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FIGURE 5.3 Reasons for Failure to Access Community Services from Team 
Member and SAP County Administrator Perspective (Source: County Administrator 
Survey) 
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Regional comparisons indicate that parent refusal is reported most often by Region 2 and 8 at a 
high of 69% and a low in Region 9 at 47%. Student refusal is highest in Region 2 and lowest in 
Region 9 at 44%. Transportation problems are highest in Regions 2 at 59% and lowest in 
Region 8 at 30%. Lack of insurance was described by 53% of respondents in Region 1, where 
Health Choices for Medical Assistance has been implemented, and lowest and in Region 2 at 
27%. 

Rural, urban and suburban schools show modest differences in reason for failure to access 
services. For all locales, parent/guardian refusal is rated highest, followed by student refusal in 
the rural and urban schools. The suburban schools indicated that transportation problems were a 
greater issue as rated by 57% of those reporting, while this was only a significant issue for 43% 
of rural schools and 34% of urban schools. It may be that time is more of a constraint on the 
parents/guardians of suburban students, than actual distance traveled. There is no significant 
variation between the scores for those indicating lack of insurance as a cause. 

Comparison of team members’ responses show some significant variation by school type as 
demonstrated in Figure 5.4. Parent refusal is the major reason that services are not accessed as 
indicated by MS (67%), HS (66%), and NPS (59%) types. This is not the dominant reason 
indicated for the AVTS population or J/SHS surveyed. Over 86% of the AVTS population 
participating in the survey cite student refusal as the significant reason, while 61% of the J/SHS 
respondents state that transportation problems are the most important reasons why services are 
not accessed. The AVTS as mentioned earlier, generally deal with an older student who 
characteristically has goals that differ from the student population in middle, J/SHS, NPS and 
HS. Teams at the J/SHS (61%) and AVTS (61%) see transportation issues as a significant 
reason why recommended services are not accessed. Transportation issues do not appear to be a 
significant issue in the NPS (12%). Perhaps this is due to the fact that there are a large number 
of K-8 grade school buildings, who deal with a significantly younger population. Parents of 
younger children may be more inclined to accommodate transportation needs in this case. 
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FIGURE 5.4 Primary Reasons for Failure to Access Recommended Services 
by School Type (N=1207) (Source: Team Member Survey) 
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A regional comparison of the county administrator responses shows little variation. For 
example, student refusal occurs most often in the Southeast, at 93%, while it is lowest in the 
Southwest at 73%. Parental refusal for a service is highest in the Central area, compared to the 
low occurring in the North at 79%. Transportation problems are highest (86%) in the Central 
and Southeast regions, and lowest in the Southwest at 73%. The Southwestern administrators 
noted denial of coverage by health insurance most often, at 46% compared to 21% in both 
Central and Southeast regions. Schedule conflicts are noted by 43% of Southeast administrators, 
while these are lowest (21%) in the North. Lack of insurance is highest in the North at 21%, 
while only 14% of Central and Southeastern administrators note this as an issue. Other reasons 
cited by administrators for failure to access recommended services include the stigma associated 
with mental health treatment (60% of the 9% total surveyed.) This is seen predominantly by 
administrators in the Central and Southwest regions. 

C. Provision of Appropriate Aftercare (follow-up) Services by the 
Contracted Providers and SAP Teams 



School -Based Support and Aftercare 

SAP teams do recommend in-school aftercare services, both for mental health and drug and 
alcohol issues. Nearly 3% of those served through SAP in 1997-98 were referred to in-school 
mental health aftercare groups and 2% were referred to drug and alcohol aftercare groups. The 
dominant in-school service, as discussed in Table 5.4 is one to one monitoring conducted by a 
school SAP team member or guidance counselor at 52% of those served. No measurement of 
aftercare service provision is made on a one to one basis, but may provide a needed perspective 
Schools may not see themselves as furnishing aftercare because they do not know they should, 
may not have the necessary staff, training or resources to provide it, or because parents and 
students may not see or desire it as an option. 
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Approximately two thirds of team members reported that follow-up services were provided to 
schools. The highest reported rate of follow-up occurs in Region 8, at 82%, while the lowest rate 
reported in Region 5 at 57%. In terms of locality, 70% of rural school team members, 69% of 
urban team members, and 59% of suburban team members related that they provided follow-up 
services. The highest rate of reported follow-up transpired at the HS level at 71%, followed by 
NPSs at 69%, AVTS at 67% and 62% at both the MS and J/SHS levels. 

Formal follow-up mechanisms include telephone contact with parent/guardian, 91%, follow-up 
meeting with parent/guardian, 68%, and follow-up letter, 33%. There is variation by region, 
with the highest reported method of follow-up being telephone contact with parents in Region 8 
at 95% and the lowest in Region 2 at 81%. The highest score for meeting as a follow-up 
mechanism was reported by Region 4 at 77% and the lowest by Region 44% for Region 5. 
Variation among the regions occurred also with the follow-up letter, as 46% in Region 5 and 9% 
in Region 4 marking this option, high and low score respectively. 



There does not appear to be much variation by locality for the telephone contact or meeting. 
Divergence does exist however with the letter, where 40% of rural school team members 34% of 
suburban and 29% of urban schools, report using this method. Variation by school type shows 
97% of AVTS, 95% of HS, 90% of MS, 89% of NPS and 85% of J/SHS use telephone contact as 
a follow-up mechanism. More NPS team members report using the meeting with parents as an 
approach at 80%, followed by 70% of AVTS, 69% of HS, 66% of J/SHS and 65% of MS. The 
only real difference for use of the letter as a follow-up strategy is at the AVTS, where 48% of 
team members reporting this modality, as compared with 35% at the J/SHS and nonpublic school 
type, while 32% at the HS and 30% at the MS use the letter. 

Community Based Support and Aftercare 

Interestingly, county administrators report that agency liaisons provide aftercare services to 
students primarily through groups. Of those surveyed, 20% report providing MH aftercare 
groups, and 18.8% report providing D&A aftercare groups. By region, mental health agencies 
are contracted for this service at about 36% with the high score in Southwestern region at 43% 
and the low in the Central region at 27%. Fifty seven percent of those surveyed in Southwestern 
region contracted with D&A agencies for this service, while only 9% of those in the Central 
region contracted with D&A agents for this service. About 17% of administrators surveyed 
contracted with both MH and D&A agencies for aftercare groups, with 29% of the Southeast 
region, 14% in the Southwest region, 10% of the Northern region and 9% of the Central region 
reported this occurring. In the Central region, 55% of respondents marked that no school-based 
aftercare services were contracted to agencies. Of the 12% of respondents marking “other,” 
100% of administrators in the Southeast use independent consultants to provide aftercare 
services in school. 
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Delineation of Desired Outcome Behaviors for Students Referred to the SAP and 

Extent of Achievement 



As discussed in Section Three, there appears to be some evidence of change in student behavior 
due to SAP as indicated by 90% of those responding to the team survey. Both the 1997-98 SAP 
Performance Report, as well as the statewide team member survey show some indication of 
alteration in student performance. 

The actual outcomes as reported in the Statewide Performance Report resulting from 
involvement in SAP include reduced disciplinary infractions for half of the referred students, 
decreased teacher concerns for 45%, and improved grades for 41 % of students referred to SAP, 
and improved attendance for 39% of referred students. In addition, less than one-third, 31%, did 
not incur any additional D&A Policy violations. Grades either improved or remained the same 
for one-third of referred students. 

SAP team members were also asked if there was evidence of change in student behavior as a 
result of SAP involvement. Almost all the team member respondents (90%) reported changes in 
student behavior as a result of the students’ involvement in the SAP program. The top four 
behavior changes evidenced: 

• Reduced disciplinary infractions (50%) 

• Decreased teacher concerns (45%) 

• Improved grades (41%) 

• Increased attendance (39%). 

There was no difference in evidence of behavior change across regions but the type of behavior 
change seen varied by locale and type of school. Rural schools more often saw no additional 
D&A policy violations (39%) compared to the urban and suburban school respondents (29%).. In 
addition, rural school respondents were more likely to see a decrease in teacher concerns (55%) 
compared to urban schools (39%) and in suburban schools (48%). 

According to school type AVTS respondents were most likely to report no additional D&A 
policy violations (69%) as compared to MS (26%), J/SHS (27%), HS (32%), and NPS (33%). 
The NPS respondents most often reported decreased teacher concern (60%) as the most common 
behavior change as compared to MS (46%), J/SHS (44%), and AVTS (35%). 
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SECTION SIX 



CONTRACT PROVIDER ASSESSMENTS 

SAP teams collect objective information on a student to make an informed decision as to the 
appropriateness of an outside referral. School student information forms generally include fact- 
finding such as academic performance, specific observable behaviors, class attendance, and 
physical behavior. Student action plans include mentoring by SAP team, referral to SAP support 
groups, referral to school-based programs, referral for mental health/drug and alcohol 
assessment, or refusal of service. Three aspects of contract provider assessments were 
investigated: (1) referral process to contract providers, (2) types of assessment utilized by SAP 
programs, and (3) contract provider assessment tools and procedures. 

SAP Referral to Contract Providers 

Contract community providers are the link for students from the educational to the appropriate 
level of care. Once a student has been recommended for an assessment, a professional MH and 
D&A liaison is contacted to determine the severity of the identified problems. This professional 
is usually a clinician with training and experience in the areas of substance abuse, child 
development, education and mental health. Primarily, licensed MH and D&A liaisons, social 
service agencies such as Children and Youth Services, and juvenile court, perform assessments. 
During these assessments both students and parent/guardians are interviewed in order to gather 
information concerning the student’s past and current functioning. At the conclusion of the 
assessment, the assessor will provide the student and family with recommendations and an 
appropriate referral for further treatment, if necessary. Referrals can be made to community 
agencies, professionals, support groups and/or in-school resources. 

Types of Assessments Utilized by SAP Programs 

As discussed in Sections 3 and 5, the types of problems referred to SAP teams are: behavior 
problems with mental health as the main concern, behavior problems with drug and alcohol as 
the main concern; suicide ideation, including gesture and attempts; and violation of school policy 
that is drug and alcohol involved. (See Table 3.4) 

SAP teams have a variety of resources available to recommend to parents. These resources can 
range from educational to psychological professionals, both within and outside of the school. 

The primary services recommended to parents by most teams are: both drug and alcohol and 
mental health agency assessments, 89%, conferences with a school counselor, 86%, and 
academic assessment 63%. Table 6.1 profiles the types of assessments recommended by the SAP 
team members. 
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TABLE 6. 1 Types of assessments used by SAP programs (Source: Team Member Survey) 



SAP Program Assessment 1 
Region totals 


Percentage 

(N=1207) 


Drug and alcohol agency assessment 


89% (1077) 


Mental health agency assessment 


89% (1072) 


School counselor conference 


86% (1032) 


Academic assessment 


63% (754) 


School psychological assessment 


60% (723) 


Multi-disciplinary evaluation 


50% (606) 


School social worker assessment 


28% (334) 


Other 


4% (48) 



School team respondents in MS, J/SHS and AVTS indicate that their programs use the following 
types of assessments: drug and alcohol, mental health, school counselors, and academic. 

Fewer NPS team members report using mental health assessments (67%), school counselor/ 
school psychologist (55%), drug and alcohol (54%), and multi-disciplinary evaluations (33%). 
Table 6.2 shows the breakdown of the type of assessments by school type. 

TABLE 6.2 Type of Assessments by School Type (Source: Team Member Survey) 



School Type 


MS 

(N=428) 


J/SHS 

(N=197) 


HS 

(N=455) 


NPS 

(N=78) 


AVTS 

(N=49) 


Drug and alcohol agency 


88% (378) 


91% (180) 


94% (429) 


54% (42) 


98% (48) 


Mental health agency 


91% (388) 


88% (174) 


91% (412) 


67% (52) 


94% (46) 


School counselor 


91% (388) 


84% (165) 


85% (388) 


59% (46) 


92% (45) 


School psychologist 


61% (262) 


61% (120) 


62% (280) 


55% (43) 


37% (18) 


School social worker 


26% (110) 


11% (22) 


40% (182) 


14% (11) 


18% (9) 


Multi-disciplinary 


49% (209) 


52% (102) 


53% (240) 


33% (26) 


59% (29) 


Academic 


60% (257) 


56% (111) 


68% (310) 


67% (52) 


49% (24) 


Other 


3% (14) 


6% (12) 


4% (18) 


4% (3) 


2% (1) 



Drug and alcohol and mental health assessments and school counselor conferences are most 
commonly available in rural, urban and suburban localities. Locality does not appear to be a 
determinant in the category of professionals performing assessments. Table 6.3 compares the 
type of assessment by type of locality. 



1 The question asked team members to indicate which services they recommend. Consequently, the number of 
responses for each item may be less than 1207 and the percentage is calculated accordingly. 



O 

ERIC 



55 



65 



TABLE 6.3 Type of Assessments Used by SAP by Locality (Source: Team Member Survey) 



Locality 


Rural 

(N=247) 


Urban 

(N=543) 


Suburban 

(N=417) 


Drug and alcohol agency 


91% (224) 


86% (467) 


93% (386) 


Mental health agency 


91% (224) 


87% (472) 


90% (376) 


School counselor 


91% (225) 


81% (440) 


88% (367) 


School psychologist 


64% (157) 


60% (326) 


58% (240) 


School social worker 


27% (67) 


36% (195) 


17% (72) 


Multi-disciplinary 


58% (142) 


52% (282) 


44% (182) 


Academic 


61% (151) 


67% (364) 


57% (239) 


Other 


2% (4) 


5% (28) 


4% (16) 



Contract Provider Assessment Tools and Procedures 

Contract provider assessments are utilized to accurately and efficiently determine the types of 
services needed. As discussed in Section One, carefully selected assessment procedures can 
quickly and validly evaluate severity of substance dependence, adverse consequences, 
contributing roles of further emotional and behavioral problems, cognitive and environmental 
stimuli. 

Many types of screening instruments are available to providers. County administrators report that 
substance abuse and mental health screenings are done by contract provider agencies using both 
informal and formal measures. Many use their own self-developed intake tool that is 
individualized for specific populations and/or situations, but the majority use more diagnostic 
instruments and standardized measures. Assessments generally include a thorough biological and 
psychosocial history including abuse (physical, emotional, and sexual), involvement with law 
enforcement/juvenile justice, drug and alcohol history, and DSM IV diagnosis if appropriate. 
Table 6.4, shows the contract provider agency assessment tools and instruments as reported by 
the County administrators. 

TABLE 6.4 County Administrator Report on Assessment Tools/Instruments Used Regularly 
by Contract Provider Agencies (Source: County Administrator Survey) 



Provider Assessment Instruments (N=53) 2 


Percentage 


Developed their own tool 


32% (10) 


Adolescent Problem/Profile Severity Index 


23% (7) 


Substance Abuse Subtle Screening Inventory 


19% (8) 


American Society of Addiction Medicine 
Adolescent Survey 


16% (5) 


Conner’s Rating Scale 


10% (4) 



2 

The question asked respondents to check all the assessment tools used by county providers. Consequently, the 
number of responses for each item may be less than 53 and the percentage is calculated accordingly. 
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Thirty-two percent (32%) of county administrators reported that the provider agencies used a 
self-developed assessment tool. Standardized assessment tools cited include the following: 
Adolescent Problem Profile Severity Index (APPSI), 23%, Substance Abuse Subtle Screening 
Inventory (SASSI) 19%, American Society of Addiction Medicine (ASAM) for adolescents, 
16%, and the Conner’s Rating Scale for mental health, approximately 10%. Additional 
instruments utilized include the Child Behavior Checklist, Index of Family Relations, Personal 
Experience Inventory, Michigan Alcohol Screening Test, Addiction Severity Index, Minnesota 
Multiphasic Personality Inventory, Alcohol and Drug Consequences Questionnaire, and 
Generalized Contentment Scale (The National GAINS Center for People with Co-Occuring 
Disorders in the Justice System, 1997). 
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SECTION SEVEN 



SCHOOL-BASED PROBATION OFFICERS 

A school-based probation officer is a juvenile probation officer with a school as his or her 
primary work site, whose authority is governed by the Juvenile Act and relevant school policies. 
The officer delivers services using the concept of the "balanced approach" to ensure community 
protection and public safety, client accountability, and the development of client competencies. 
As one of his or her duties, each SBPO is required to be SAP trained. It is expected that each 
SBPO will be a member of the SAP team and that, at a minimum, he or she will attend all 
meetings related to probation clients. Investigated in year one was the school based probation 
officer SAP involvement. 

School Based Probation Officer SAP Involvement 

Data on the school based probation officers program was collected through the team member and 
county administrator surveys and the site visits. The results indicate that 22% of the team 
members surveyed report that a school based probation officer participates on the SAP team. The 
PCCD indicated that during the 1998-1999 school year, 141 out of the 501 school districts 
(28 %) in the Commonwealth have access to school based probation officer involvement in 
schools. 

Examining the response patterns of the team members who reported school based probation 
officer participation reveals a consistent view of the officers’ tasks. Table 7. 1 profiles the tasks 
of these officers. The primary tasks include monitoring student behavior, 74%, participating in 
team meetings, 53%, and providing one to one counseling, 48%. 

TABLE 7. 1 Primary Tasks Performed by School-Based Probation Officers (Source: Team 
Member Survey) 



Task 


Percentage 
(N = 269) 


Monitors student behavior 


74% (199) 


Participates in team meetings 


53% (142) 


Provides one to one counseling 


48% (128) 


As needed 


32% (85) 


Refers students to SAP 


22% (60) 


Informal contact 


20% (55) 


Monitors student grades 


19% (52) 


As invited by team 


13% (36) 


Probation officer contacts team 


10% (26) 


Telephone conversation 


9% (24) 


Provides group counseling 


6% (17) 


Other 


4% (12) 
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The above section provides a description of the tasks performed by the probation officer from the 
perspective of the team members. The SBPO’s tasks were also examined from the perspective of 
the ten probation officers who responded to the survey and may not be reflective of all probation 
officers that sit on SAP teams. When asked to cite the most important tasks of the SBPO on the 
SAP team, the responses corresponded with the results in the team survey: monitoring behavior 
and attending team meetings. 

Likewise, the responses of the county administrators corresponded to the same activities of the 
school-based probation officers as reported by the team members. As indicated in Table 7.2, the 
county administrators were asked to indicate ways school based probation officers were involved 
with SAP teams. The administrators concur that monitoring behavior and participating in team 
meetings were the top two SBPO tasks. 

TABLE 7.2 County Administrator Perceptions of School-Based Probation Officer Tasks 
(Source: County Administrator Survey) 



Task 


Percentage 
(N = 37) 


Monitors student behavior 


70% (26) 


Participates in team meetings 


68% (25) 


Refers students to SAP 


60% (22) 


Telephone conversation 


60% (22) 


Informal contact 


60% (22) 


As needed 


60% (22) 


Monitors student grades 


54% (20) ~ ~ ] 


Provides one to one counseling 


38% (14) 


Provides group counseling 


22% (8) 
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SECTION EIGHT 



SUGGESTED IMPROVEMENTS AND SATISFACTION 

A valuable source of information and suggestions on how to improve SAP are the many SAP 
stakeholders. Two sources of potential feedback for SAP were examined: (1) team members’ 
and county administrators’ suggestions to improve SAP and (2) methods to assess satisfaction 
with SAP. 



SAP Improvement 

In both statewide surveys, an open-ended question was included asking the team members and 
county administrators to offer their suggestions for improving SAP. A small proportion of 
respondents provided specific suggestions. The responses were categorized into the following 
areas: public awareness, training and in-services, team structure, funding, administration, parent 
involvement, agency and community issues, and service provision. Table 8.1 profiles the top 
four areas of suggestions provided by the team members and the county administrators. Team 
members and administrators agreed that team structure and public awareness are priority 
improvement areas. With regard to team structure, specific suggestions offered included size, 
diversity, and number of team members who were teachers and administrators. Those 
respondents whose teams who did not have a full-time coordinator suggested that having one 
would improve their effectiveness. 

Public awareness was also viewed as an important factor for SAP success. For example, 
suggestions included to increase the awareness of superintendents and school board members 
regarding the importance of SAP as well as to conduct public relations efforts for all members of 
the school community. County administrators added that education of the community on mental 
health and drug and alcohol issues and its relationship to SAP would further strengthen the 
system. Liaison and team member focus groups agreed that SAP visibility was a priority. 
Another suggested area for improvement was training and in-service, which included yearly 
introductions to SAP, faculty in-services, team maintenance, group facilitator training, quarterly 
training and refresher courses. Specific training for new team members and central office 
administrators was also seen as critical. Team members stressed that a mechanism for 
differentiating between appropriate and inappropriate referrals was essential. 

An additional area consisted of agency and community issues, including the need for increased 
liaison services or an improved communication system for existing services. For example, team 
members recommended more school-based probation officer involvement and participation at 
SAP meetings. Participants at the site visits indicated that consistent notification of student 
probation involvement would enhance their ability to provide effective services. County 
administrators suggested the need for consistent and updated guidelines for both and MH and 
D&A liaisons. Team members concurred with the need for formalizing procedures and having 
clear guidelines for liaisons. Some site visit participants cited the need for greater access to 
community resources, particularly with D&A liaisons. County administrators noted team 
structure as a priority. This includes enough staff, counselors, and coordinators for the SAP 
process to be accomplished. This was also interpreted to mean adequate time for the team, as 





well as the direct service providers, to accomplish their tasks. The surveys endorsed the need to 
provide a reliable common meeting time, to prepare for emergency meetings, and consistency for 
the complete team to meet. 

County administrators indicated that consistent and increased funding was a priority area in need 
of improvement. Responding team members also cited this as an area for improvement. Specific 
suggestions included funding for continued training, additional drug and alcohol and mental 
health services, as well as increased insurance coverage for adolescents. These areas were also 
identified at the site visits and focus groups. County administrators also suggested that more 
funding is needed to increase teacher release time, have full-time rather than part-time help, and 
increase both in-school and community-based services. Further, they also indicated the need to 
reorganize the funding system that services SAP; for example, combining the mental health and 
drug and alcohol streams in terms of training, funding, paperwork and requirement guidelines. 

County administrators cited the need for more support and involvement from school 
administrators as a top priority. Responding team members also suggested that administrative 
involvement was an area for improvement. More district commitment, more representation, and 
more participation on the team were the persistent observations made by team members, focus 
groups, and site visits. 

Table 8.1 Primary Areas of Improvement Suggested by Team Members and County 

Administrators (Source: Team Member and County Administrator Surveys) 



Improvement 


Team Member 


County Administrator 


Team Structure 


X 


X 


Funding 


— 


X 


Administrator Involvement 


— 


X 


Public Awareness 


X 


X 


Training and In-Service 


X 


-- 


Agency and Community Issues 


X 


— 



Client Satisfaction 



Client satisfaction provides one perspective of service quality. Results from the statewide team 
survey indicate that of the entire sample of team members, 26% reported collecting client 
satisfaction data. Figure 8.1 shows the primary methods utilized by the responding team 
members for assessing satisfaction. They include parent interview (63%), student interview 
(56%) and telephone calls (35%). Other methods consisted of student questionnaires (25%) and 
parent/guardian questionnaires (9%). 
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FIGURE 8. 1 School Methods for Assessing Client Satisfaction with SAP 
(Source: Team Member Survey) 
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Participants at two of the sites visited indicated that they have a mechanism in place for assessing 
client satisfaction. One mechanism used was focus groups with students to assess their 
satisfaction with SAP and generate ways to make it more effective. In addition, during 
scheduled parent-community round table discussions convened by the district to investigate 
many issues, parents were asked about their knowledge and understanding of SAP. Another 
method was exit interviews with students to assess satisfaction with a SAP-recommended 
program. For example, in using the above method it was indicated that the students do not 
necessarily like SAP, but would recommend it to friends having difficulty and that the program 
be continued. 

County administrators were also asked if they required provider agencies to assess parent 
satisfaction with SAP. Of the 42 SAP county administrators who contract with community 
agencies to provide SAP services, 69% of respondents do not require their service providers to 
assess parent satisfaction with SAP. Approximately 64% of administrators reported that they do 
not ask the provider agencies to assess student satisfaction with SAP; however, 62% require that 
the agency assess school staff satisfaction with SAP. 

Of the eleven SAP county administrators that provide direct services to SAP teams, six of the 
administrators report assessing student satisfaction with SAP, and three report assessing parent 
satisfaction. In comparison, nine of the SAP county administrators indicated that they conduct 
school staff satisfaction surveys with SAP. 



SECTION NINE 



LIMITATIONS OF THE FIRST YEAR EVALUATION FINDINGS 

Several data sets were compiled for this evaluation that have been utilized in providing both 
profile and comparative analyses to determine the effectiveness of SAP in Pennsylvania. They 
include the following: 

1 . a description of the functioning of SAP teams from the perspective of the 
approximately 1200 team members as well as 48 out of 53 county administrators 
including the referral process and the intervention services provided 

2. the effect of parent and student consents on involvement and SAP 

3. the nature of the relationship between the SAP programs in schools and the counties, 
particularly the types of assessments utilized by contract providers 

4. the role of school based probation officers on SAP 

5. suggestions for improvement of SAP 

6. mechanisms for assessing client satisfaction with SAP, and 

7. the impact of involvement in SAP in the areas of school performance as documented 
by a school designated team member and submitted to the statewide performance 
database. 

These various data sources provide a comprehensive depiction of SAP; however, there are four 
limitations to this evaluation, which can be addressed in future evaluations. First, the results are 
presented from the perspective of the team members and the county administrators who are the 
service monitors and/or providers. They do not include an assessment of SAP from the 
viewpoint of the service recipients, that is the students and/or the parents whose children have 
been involved in SAP. This could only be accomplished if there were a mechanism for 
interviewing or surveying parents or students, or to access data from student records. This 
creates additional challenges due to confidentiality issues as well as sensitivity in some school 
systems to allow student based research in their districts. 

Second, this study does not present an indication as to the extent to which various indicators of 
the effectiveness of SAP are occurring at appropriate levels. For example, what is an appropriate 
rate of referral to SAP? This type of analysis could be conducted by: reviewing the literature to 
determine the expected prevalence in schools for the types of problems referred to SAP, meeting 
with teams to determine their expectations of an appropriate rate of referral; analyzing existing 
data sources to measure the rate of referral, and subsequently develop benchmarks for rates of 
referral. Such information is essential in determining whether SAP is functioning appropriately, 
but more importantly, providing teams and administrators with data that has the potential of 
being employed as a gauge for the continuous improvement of SAP. 

Third, the statewide performance database is comprised of information compiled and completed 
by team members at the end of the school year. It only consists of post- SAP measures and does 
not allow for an adequate comparison of the change in student performance that may be 
attributed to involvement in SAP. For example, it is not clearly documented as to whether the 
data submitted compares a student’s actual grades prior to receipt of SAP services with grades 
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subsequent to the receipt of SAP services. In some instances teams may examine school records, 
but in others they may be relying on other sources such as teacher, parent or student report. 

In addition, the data submitted does not clearly document how long a student has been involved 
in SAP. The report provides aggregate numbers with regard to the number of students receiving 
services from SAP but does not delineate the length of time a student has been receiving such 
services. In some instances, a student may have been involved in SAP since the beginning of the 
school year while another may have entered in April, but they are both counted as new referrals. 
There is no mechanism to link length of time in SAP with outcome. 

Fourth, each of the above-mentioned data sources consist of disparate units of analysis, 
restricting the ability to link databases critical to assessing the overall impact of SAP on student 
performance. For example the unit of analysis for the statewide team survey is team member in 
a school building, while the unit of analysis in the statewide student performance database is the 
student in the school building as described by the team member. To appropriately link 
databases, a common unit of analysis needs to be identified, which in this instance is the school 
building. A methodology comprised of specific rules must be developed to generate a set of new 
variables at the school building level using individual level data. An illustration of how this can 
be constructed is as follows. 

Suppose we want to examine the relationship between the effects of parental consent (SAP Team 
Member Statewide Survey) on the student performance outcome measures (Statewide 
Performance Database). Two new variables at the school building level have to be constructed. 
First, we need to calculate the percentage of SAP members within each school building who 
indicate that they obtained written informed consent before the SAP intervention. An index on 
the level of receipt of informed consent will be generated for each school building based on the 
following formula (80-100%=High, 60-79%=Medium, and under 60%=Low). Second, we need 
to generate an index of student improvement in academic performance for each school building 
using student performance data from the Statewide SAP database. For example, 80-100% of 
students who show improvement or remained the same due to a SAP intervention in a school 
building will yield a “High” performance score, 50-79% will receive a “Medium” score, and 
under 50% would be rated “Low.” Finally, using the new variables generated above, we can 
conduct an analysis of the impact of the level of consent to predict academic performance. A 
complete documentation of the methodology utilized will be provided to PCCD, PDE and other 
appropriate parties. 
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SECTION TEN 



SUMMARY, CONCLUSIONS AND RECOMMENDATIONS 

As previously described, the aim of the evaluation was to examine the effectiveness of SAP 
throughout the Commonwealth of Pennsylvania. A variety of data sources have been developed 
to provide a comprehensive description of the various components of and/or relating to SAP. 
The following section provides a summary of preliminary findings and conclusions that 
potentially have implications for policy and practice. 

The results of the evaluation of the various data sets indicate that the SAP model in 
implementation and practice closely corresponds with the core team model as it was originally 
designed. This includes adherence to the four phases of the SAP process, that is referral, team 
planning, intervention and recommendation, and support and follow-up. However, local 
implementation varies according to school structure and policy, as well as community needs and 
resources. For example, parental consent practices may vary depending upon the age of the 
student, severity of problem, team size and community resources. 

The findings also suggest that the implementation of this model was supported via participation 
in competency based training by team members, administrators and other relevant staff. In 
addition, results indicate that the teams are functioning, as they meet at least one time each week 
and there is a defined leadership structure. 

When examining the six areas investigated, the primary findings as summarized below indicate 
that the SAP teams are working, that parents are involved, services are being provided both in 
school and in the community, and that the students involved in SAP are showing definite 
improvement. However, given the time period for gathering these data, further analyses that 
would link process with outcome is recommended for future study. The following provides a 
description of the major conclusions and recommendations evolving from this evaluation for 
each of the areas under investigation. 



Referral Process 



A. Summary and Conclusions 

Results indicate that the referral process is functioning and suggest it may be effective for 
students referred to SAP. This is verified in the following areas: 

• Five primary methods are used to publicize the SAP referral process: faculty in- 
service, student handbook, brochure, faculty handbook and classroom visit; 

• Multiple mechanisms are in place for making referrals, including a referral form, 
informal meeting, telephone conversation and formal meetings; 

• Most teams have established acceptance criteria, either informal or formal; 

• A team approach is employed to determine the appropriateness of a referral and the 
majority of teams provide feedback to the referring entity, including verbal contact, 
letter, or telephone call; 
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• SAP teams address a wide range of barriers to learning including mental health 
behavioral concerns, drug and alcohol behavioral concerns, academic and attendance 
problems; 

• Team members expect that there will be a positive outcome from involvement in SAP 
including increased family communication, no further drug and alcohol policy 
violations and reduced disciplinary infractions. These outcomes relate to engagement 
into the SAP process. 

B. Recommendations 

• Devise formal referral/acceptance criteria to give consistent guidelines to teams; 

• Incorporate information from national literature, statewide data sources and local 
school related statistics to determine appropriate benchmarks for rate and type of 
referral. 

Student and Parent Consent Practices 
A. Summary and Conclusions 

• Teams are proactive in attempting to engage student and parent consent in SAP as 
evidenced by 86% of parents who are contacted; 79% of team members report 
obtaining the written informed consent of parents prior to student involvement in 
SAP; 

• The majority of teams contact parents after gathering some initial objective 
information about the students’ performance in school; 

• Primary methods of contact include telephone call, letter and meeting; 

• Recommended services are being accessed as evidenced by 61% of parents and 
students reported has having accessed them; 

• Primary reasons that parents and students do not access recommended services 
include parental refusal, student refusal, denial that a problem exists, privacy issues 
and mental health stigma. 

B. Recommendations 

• Develop a parent oriented public awareness campaign to address issues such as 
privacy, denial, and mental health stigma; 

• Develop a model based upon documented research for assisting schools in choosing 
the most effective methods for increasing parent participation; 

• Enhance existing training for SAP teams that assists them in addressing denial and 
privacy issues, as well as the mental health stigma. 
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Intervention Services 



A. Summary and Conclusions 

The primary actions taken by SAP teams include recommendation for assessment, 
continued monitoring by SAP team, conference with the school, the parent and the 
student, and referral to MH andD&A agency; 

In-School services recommended by SAP teams include follow-up with a team 
member or counselor, intervention groups and academic supports; 

D&A and MH liaison services are provided to teams at least one day per week; 

The majority of counties contract with community agencies to provide liaison 
services to schools, however, approximately 20-25% of the counties provide direct 
services to the schools; 

Approximately two thirds of students and parents access the services recommended 
by the SAP team; 

However, there is limited involvement of SAP teams in treatment and aftercare plans 
due to parental refusal of consent to share information, failure to obtain consent from 
parents, and agency caseload volume; 

SAP team members and the county administrators revealed five primary reasons for 
not accessing recommended community services. They include, parental refusal, 
student refusal, transportation problems, lack of insurance and denial of coverage by 
health insurance; 

The primary expected outcomes of SAP involvement as reported by the team 
members are increased communication with the family, no further D&A policy 
violations, reduced disciplinary infractions, advancement to the next grade level, 
increased attendance, and improved grades; 

The most positive outcomes as recorded in the statewide performance database are 
reduced disciplinary infractions, decreased teacher concerns, improved grades, and 
improved attendance; 

Aftercare services are provided in school by agency liaisons primarily in group 
format for both mental health and drug and alcohol issues. 

B. Recommendations 

Develop guidelines for D&A liaisons to parallel the MH liaison guidelines; 

Educate the treatment community in the importance of and strategies for involving 
SAP teams in treatment and follow-up planning. 

Assessments Conducted by Contracted Providers 

A. Summary and Conclusions 

Licensed contracted providers are conducting assessments for identified drug and 
alcohol and mental health referrals; 

County administrators report that a substantial number of the contracted provider 
agencies have developed their own screening/assessment tools and instruments; 



• There is wide variation in the types of standardized instruments reported by county 
administrators as being used which include the Adolescent Problem/Profile Severity 
Index (APPSI), the Substance Abuse Subtle Screening Inventory (SASSI), the 
American Society of Addictions Medicine (ASAM) Adolescent Survey and the 
Connors Rating Scale; 

• Contractual arrangements with licensed agencies for the provision of SAP services 
are in place in the majority of counties/county joinders. 

B. Recommendations 

• Identify the types of assessment tools actually used by the contract providers; 

• Determine which types of tools are most appropriate for screening and which are 
most appropriate for designating level of care. 

School Based Probation Officer Involvement in SAP 
A. Summary and Conclusions 

• SAP teams that report having a probation officer assigned to their school consistently 
report that the officer contributes to the SAP process; 

• The primary tasks of school based probation officers are monitoring student behavior, 
participating in SAP meetings, and counseling on a one-to-one basis. 

B. Recommendations 

• Increase the number of schools who have access to school based probation officers; 

• Increase the number of school based probation officers who are actively involved in 
SAP; 

• Develop guidelines for involving school based probation officers in SAP. 

Suggestions for Improvement and Client Satisfaction 
A. Summary and Conclusions 

• County administrators report team structure, funding, administrative involvement and 
public awareness as significant areas for SAP improvement; 

• SAP team members report training and in-service, team structure, agency and 
community issues, consistent source of funding and public awareness as significant 
areas for SAP improvement; 

• Additional suggestions for SAP improvements include improvements in the areas of 
parental involvement and service provision; 

• The assessment of parent and student satisfaction is not a current practice for SAP 
teams. Presently, less than 25% of teams report measuring parent and student 
satisfaction with SAP; 
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• Assessment of satisfaction is not required by the county entities that fund SAP service 
provision to students and parents. However, the majority of county administrators report 
that the contracted agencies assess school staff satisfaction with SAP. 

B. Recommendations 

• Develop practice guidelines for assessing parent and student satisfaction with SAP; 

• Develop tools for assessing parent and student satisfaction with SAP; 

• Develop a reporting mechanisms that solicit suggested improvements from the field. 

Final Conclusions 



In examining the overall effectiveness of SAP, the following final conclusions emerged. First, a 
defined well-communicated referral process is operational in most of the teams, and the potential 
referrers know how to access and utilize the student assistance team. Second, SAP teams contact 
and involve parents in the process. In cases where parents refuse SAP services, the primary 
reasons include denial that a problem exists and privacy issues. Third, SAP teams are making 
recommendations for a variety of services including assessments and are monitoring students on 
an ongoing basis. Fourth, D&A and MH liaison services are in place providing preliminary 
assessment and linkage with service providers. Fifth, an increasing number of schools are 
receiving services from school based probation officers. Their involvement in SAP is evolving in 
accordance with the changing times in school settings. These officers have specialized 
knowledge of a limited student population and initially their participation was limited to their 
professional role and function. Currently their role is expanding to become a skilled resource to 
the team. Sixth, while currently not extensively practiced, the assessment of parent and student 
satisfaction has the potential to contribute to the continued quality improvement of SAP. 

Seventh, similar suggestions for improvement of SAP were provided by team members and 
county administrators, however, their prioritization of suggestions reflects their role in the SAP 
process. 
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SECTION ELEVEN 



IMPLICATIONS FOR FURTHER EVALUATION OF SAP IN PENNSYLVANIA 



The findings discussed in the previous sections provide a comprehensive descriptive and 
comparative profile that sheds light on several key questions about Student Assistance Programs 
in Pennsylvania. This section presents a number of strategies that could be employed to 
systematically measure the program’s effect on student outcomes. 



Development of Benchmarks and Indicators to Guide Effective SAP Teams and Programs 

The development of benchmarks and indicators is crucial to the continued improvement of 
services to students through SAP and should be structured to delineate different categories of 
students based upon student needs and risk factors. Based upon the results, it is clear that the 
teams are functioning but there is not a system in place for actually measuring their effectiveness. 
In other words, they collect information such as changes in student performance at the end of the 
year, but do not have a method for comparing their actual results with what can be realistically 
expected. By developing appropriate benchmarks and indicators, the teams will be able to gauge 
their effectiveness by comparing their results with a standard developed from this evaluation. 

For example, a team may receive 130 referrals in a school of 1300 students, for a rate of 10%. 
The development of an appropriate benchmark will show whether that referral rate exceeds, 
meets, or is below the standard. Such information can be used by the team to determine whether 
changes are needed in educating parents and students or faculty and administration about SAP, 
by the school in identifying other mechanisms for improving referrals, and by the state to more 
effectively monitor referrals to teams throughout the Commonwealth. 

The benchmarks and indicators can be developed from the data sources employed in this 
evaluation. Site visits could be conducted on a random sample of teams throughout the state. 

The purpose would be to delineate how the teams work; to discover what, if any, indicators are 
currently being used to monitor effectiveness; and to obtain more comprehensive qualitative 
profiles of SAP. 



Upon completion of this phase of the evaluation, the benchmarks and indicators can be piloted in 
each of the five types of schools: HS, J/SHS, MS, AVTS, and NPS. The purpose will be to test 
the practical application of the benchmarks and indicators and make appropriate revisions to 
them prior to the final development of the state-of-the-art manual and implementation of 
statewide training. 
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Technology Transfer 



Technology transfer can occur at two levels. First, a state-of-the-art manual could be developed 
and would describe how the benchmarks and indicators were derived, how teams can gather the 
information needed for comparative purposes, and the practical application of benchmarks and 
indicators for the continued improvement of SAP at the school, school system, county, and state 
levels. It could also include benchmarks for specific categories that are based upon risk factors 
gleaned from national statistics. Second, technology transfer could occur through formal training 
to target audiences. Training of trainers within the existing state training mechanisms such as the 
Commonwealth Approved Training System (CATS), and at appropriate statewide conferences. In 
addition, a separate track of training could be developed for policy makers, legislators and other 
stakeholders to assist them in informed decision making. 
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